Re: groupby behaviour
On Tue, Feb 26, 2013 at 9:27 AM, andrea crotti andrea.crott...@gmail.com wrote: So I was trying to use groupby (which I used in the past), but I noticed a very strange thing if using list on the result: As stated in the docs: The returned group is itself an iterator that shares the underlying iterable with groupby(). Because the source is shared, when the groupby() object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as a list: -- http://mail.python.org/mailman/listinfo/python-list
Re: groupby behaviour
2013/2/26 Ian Kelly ian.g.ke...@gmail.com: On Tue, Feb 26, 2013 at 9:27 AM, andrea crotti andrea.crott...@gmail.com wrote: So I was trying to use groupby (which I used in the past), but I noticed a very strange thing if using list on the result: As stated in the docs: The returned group is itself an iterator that shares the underlying iterable with groupby(). Because the source is shared, when the groupby() object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as a list: -- http://mail.python.org/mailman/listinfo/python-list I should have read more carefully sorry, I was in the funny situation where it would have actually worked in the production code but it was failing in the unit tests (because I was using list only there). It's very weird though this sharing and still doesn't really look rightl, is it done just for performance reasons? -- http://mail.python.org/mailman/listinfo/python-list
Re: groupby behaviour
andrea crotti andrea.crott...@gmail.com writes: It's very weird though this sharing and still doesn't really look rightl, is it done just for performance reasons? It could consume unbounded amounts of memory otherwise. E.g. if there are millions of items in the group. If you're not worried about that situation it's simplest to just convert each group to a list for processing it: for k,g in groupby(...): gs = list(g) # do stuff with gs calling list(g) reads all the elements in the group, advancing groupby's internal iterator to the next group. Since you've copied out all the group elements, you don't have to worry about the sharing effect. -- http://mail.python.org/mailman/listinfo/python-list
