Re: [Numpy-discussion] very simple iteration question.
On Wed, 2008-04-30 at 21:09 +0200, Gael Varoquaux wrote: On Wed, Apr 30, 2008 at 11:57:44AM -0700, Christopher Barker wrote: I think I still like the idea of an iterator (or maybe making rollaxis a method?), but this works pretty well. Generally, in object oriented programming, you expect a method like rollaxis to modify an object inplace. At least that would be my expectation. BTW. rollaxis isn't a method. I was completely unaware of this function. Learned something new today... BC Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
On Wed, Apr 30, 2008 at 3:09 PM, Gael Varoquaux [EMAIL PROTECTED] wrote: On Wed, Apr 30, 2008 at 11:57:44AM -0700, Christopher Barker wrote: I think I still like the idea of an iterator (or maybe making rollaxis a method?), but this works pretty well. Generally, in object oriented programming, you expect a method like rollaxis to modify an object inplace. At least that would be my expectation. Gaël, I'm not sure where you learned this expectation, as I don't think (but I could be wrong -- so educate me!) it is universal for OOP nor encouraged for Python in particular. OOP promotes encapsulation by providing mechanisms for objects to publish an interface and thereby hide the details of their inner workings, but I don't think there is a common OOP philosophy that disallows methods returning views and requires all transformations be performed in-place. A design philosophy like you've espoused is tractable in languages that support overloading, like C++, but it is not tractable in Python (at least not cleanly within the design of the base language). How do you create a function that returns a flat iterator of a container generically? In C++, each container would overload the flat function. In Python, your only hope is to ensure that the flat function only requires a standardized interface to a container object, but that would likely depend on the standardized interface including some low level iteration method from which to build more complex iterators. Even in C++ where such a design philosophy is possible at some level, objects often expose const iterators of various sorts as methods. Finally, Python has plenty of counter-examples to the maxim: all the string methods because strings are designed to be immutable, set object methods (even for mutable sets), various iterators for dicts, etc. On this topic. I would love to see numpy evolve a moderately generic array interface so that we could write stand-alone functions that work generically with ndarrays, as well as masked arrays and sparse arrays so that there was one dot-product to rule them all, so to speak. Right now, you can't use functions like numpy.dot on a numpy.ma.MaskedArray, for example. Regards, Alex ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
for i in range(52): week_data = data[:,i,:] OR for week_data in data.transpose(1,0,2): ... Nadav -הודעה מקורית- מאת: [EMAIL PROTECTED] בשם a g נשלח: ד 30-אפריל-08 11:11 אל: numpy-discussion@scipy.org נושא: [Numpy-discussion] very simple iteration question. Hi. This is a very basic question, sorry if it's irritating. If i didn't find the answer written already somewhere on the site, please point me to it. That'd be great. OK: how do i iterate over an axis other than 0? I have a 3D array of data[year, week, location]. I want to iterate over each year at each location and run a series of stats on the columns (on the 52 weeks in a particular year at a particular location). 'for years in data:' will get the first one, but then how do i not iterate over the 1 axis and iterate over the 2 axis instead? thanks, alex. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
2008/4/30 a g [EMAIL PROTECTED]: Hi. This is a very basic question, sorry if it's irritating. If i didn't find the answer written already somewhere on the site, please point me to it. That'd be great. OK: how do i iterate over an axis other than 0? I have a 3D array of data[year, week, location]. I want to iterate over each year at each location and run a series of stats on the columns (on the 52 weeks in a particular year at a particular location). 'for years in data:' will get the first one, but then how do i not iterate over the 1 axis and iterate over the 2 axis instead? Well, there's always for i in xrange(A.shape[1]): do_something(A[:,i,...]) But that's kind of ugly in this day and age. I would be tempted by for a in np.rollaxis(A,1): do_something(a) It's worth mentioning that traversing an array along an axis not the first usually results in subarrays that are not contiguous in memory. While numpy makes working with these convenient, they may not be very efficient for cache reasons (for example, modern processors load from memory - an extremely slow operation - in blocks that may be as large as 64 bytes; if you only use eight of these before moving on, your code will use much more memory bandwidth than it needs to). If this is a concern, you can usually transparently rearrange your array's memory layout to avoid it. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
Hi Alex, a g wrote: Hi. This is a very basic question, sorry if it's irritating. If i didn't find the answer written already somewhere on the site, please point me to it. That'd be great. You should look at any of the documents below and read up on array slicing. It is perhaps the most important and pervasive concept of Numpy and should be understood by all users. Numpy Tutorial: http://www.scipy.org/Tentative_NumPy_Tutorial Numpy for MATLAB users: http://www.scipy.org/NumPy_for_Matlab_Users Guide to Numpy OK: how do i iterate over an axis other than 0? I have a 3D array of data[year, week, location]. I want to iterate over each year at each location and run a series of stats on the columns (on the 52 weeks in a particular year at a particular location). 'for years in data:' will get the first one, but then how do i not iterate over the 1 axis and iterate over the 2 axis instead? It is not clear to me whether you want to slice or iterate over an array. Assuming you are fixing the year and location, the following code iterates over data for fixed year and location. for week in xrange(0, 52): do something with data[year, week, loc] Slicing is more efficient and you should use it if you can. Fixing the year and location, the following computes the mean and standard deviation across all weeks. All of the statements below yield scalars. data[year, :, loc].mean() -- takes the mean of the data across weeks data[year, :, loc].std() -- takes the standard deviation of the data across weeks You should download IPython and type help(numpy.array) to see one set of functions you can call on the result of a slice (sum, min, etc.). Although I don't know what statistics you are computing for sure, the following code might be useful since it computes a statistic across all weeks for each year and location value. data.mean(axis=1) It yields a num_years by num_locations array mu where mu[y, l] is the average data value across all weeks for year y and loc l. I hope this helps. Damian ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
a g wrote: OK: how do i iterate over an axis other than 0? This ties in nicely with some of the discussion about interating over matrices. It ahs been suggested that it would be nice to have iterators for matrices, so you could do: for row in M.rows: ... and for column in M.cols: ... If so, then wouldn't it make sense to have built in iterators for nd-arrays as well? something like: for subarray in A.iter(axis=i): ... where axis would default to 0. This is certainly cleaner than: for j in range(A.shape[i]): subarray = A[:,:,j,:] Wait! I have no idea how to spell that generically -- i.e. the ith index, where i is a variable. There must be a way to build a slice object dynamically, but I don't know it. How often would i be a variable, rather than hard coded? I have no idea. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
2008/4/30 Christopher Barker [EMAIL PROTECTED]: a g wrote: OK: how do i iterate over an axis other than 0? This ties in nicely with some of the discussion about interating over matrices. It ahs been suggested that it would be nice to have iterators for matrices, so you could do: for row in M.rows: ... and for column in M.cols: ... If so, then wouldn't it make sense to have built in iterators for nd-arrays as well? something like: for subarray in A.iter(axis=i): ... where axis would default to 0. This is certainly cleaner than: for j in range(A.shape[i]): subarray = A[:,:,j,:] Wait! I have no idea how to spell that generically -- i.e. the ith index, where i is a variable. There must be a way to build a slice object dynamically, but I don't know it. Slices can be built without too much trouble using slice(), but it's much easier to just write for subarray in np.rollaxis(A,i): ... rollaxis() just pulls the specified axis to the front. (It doesn't do what I thought it would, which is a cyclic permutation of the axes). Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] very simple iteration question.
On Wed, Apr 30, 2008 at 11:57:44AM -0700, Christopher Barker wrote: I think I still like the idea of an iterator (or maybe making rollaxis a method?), but this works pretty well. Generally, in object oriented programming, you expect a method like rollaxis to modify an object inplace. At least that would be my expectation. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
