[issue17862] itertools.chunks(iterable, size, fill=None)
Raymond Hettinger added the comment: Nothing new is happening in this thread, so I'm closing it for the reasons listed in the other posts. The main issue is that the generic concept of break data into chunks tends to occur is situations where the iterator protocol would be at odds with a clean solution. A reshape() method on lists would be much better suited to the task. -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Changes by Josh Rosenberg shadowran...@gmail.com: -- nosy: +ShadowRanger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Raymond Hettinger added the comment: If this is to go forward, it needs to be more interesting, useful, and general than what was has been discussed so far. I would be open to some kind of reshape() itertool than can ungroup, flatten, and regroup in at least two dimensions. Ideally, it should be inspired by a successful general-purpose tool from another functional or data manipulation language (perhaps APL, Mathematica, Matlab, Numpy, or somesuch). Ideally, the proposal will be accompanied by some non-trivial real-world use cases to help validate the design. Ideally, there should be demonstrations of reshape() interacting effectively with the other itertools (i.e. a criterion for adding new Lego bricks is whether they work well with all the existing Lego bricks -- that is what makes a good Lego set). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Serhiy Storchaka added the comment: We should distinguish between at least two different functions. One generates slices of input sequence (it is especially useful for strings and bytes objects), and other groups items from arbitrary iterator into tuples. They have different applications. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Jakub Stasiak added the comment: I'm just gonna leave my implementation of chunk function (not sure about the name yet) here, it's basically what itertools.chunks from the previous patch is but it works for arbitrary iterables + few tests and documentation. The last chunk is currently truncated if there are not enough elements to fill it completely. -- nosy: +jstasiak Added file: http://bugs.python.org/file31565/itertools.chunk.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Serhiy Storchaka added the comment: A week ago I implemented chunks() on C for issue17804. This is an equivalent of such Python code for unlimited sequences: def chunks(seq, size, start=0): for i in itertools.count(start, size): yield seq[i: i + size] or simpler for limited sequences: def chunks(seq, size, start=0): for i in range(start, len(seq), size): yield seq[i: i + size] Later I gave up the idea when I saw the insignificance of the benefits. Personally I have such arguments against including it in stdlib: 1. While C implemented chunks() is faster than manual iteration, speed up of real loops is not worth the use of special function. 2. This idiom is used less than I expected (about two dozen times in stdlib, not counting tests and tools) and use chunks() saves too little number of lines. In any case Python implementation is only 2-3 lines. 3. This function is not very well suited for the itertools module, because it works with sequences and not with iterators. -- keywords: +patch nosy: +serhiy.storchaka Added file: http://bugs.python.org/file30177/iter_chunks.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Roundup Robot added the comment: New changeset 763d260414d1 by Raymond Hettinger in branch '2.7': Issue 17862: Improve the signature of itertools grouper() recipe. http://hg.python.org/cpython/rev/763d260414d1 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Roundup Robot added the comment: New changeset 6383d0c8140d by Raymond Hettinger in branch '3.3': Issue 17862: Improve the signature of itertools grouper() recipe. http://hg.python.org/cpython/rev/6383d0c8140d -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Raymond Hettinger added the comment: The reasons for the previous rejections still hold: more tools make the overall toolset harder to use, not being a primitive operation, dearth of real-world use-cases, not being prevalent in other functional languages, easily expressible with existing tools, non-consensus on what to do with odd-sizes, lack of adoption of the published recipe, and a hard-to-guess function name. In addition to previously listed reasons, I have vague feelings that this isn't the right thing to do. The feelings are in-part based on poor user experience with itertools.groupby(), a function that developers said they wanted but ended-up being awkward to fit into real applications, confusing to some users, and rarely used in practice. Another source of misgivings is that iterators may not be the right tool for this kind of task. For example, when partitioning data into subgroups for a map/reduce operation, iterators won't help because they are evaluated serially which precludes any chance of parallelization. Even in cases of serial processing such reading blocks from a socket, the chunks iterator would be useless or awkward (i.e. we need more versatility than iterator.next() to manage the control-flow, time-outs, out-of-band control, separating headers from content, etc.) In other words, I have a sense that the generic concept of break data into chunks tends to occur is situations where the iterator protocol would be at odds with a clean solution. That said, I'll leave this open for a while and do my best to try to warm-up to it. Your recurring enthusiasm for it is a positive point. Another is its the faint resemblance to a numpy reshape operation. P.S. In prior discussions, the only real use case that ever surfaced was printing long sequences of data across multiple columns. Even that use case was suspect because the desired order is typically in column-major order (for example, look at the output of the *nix ls command). -- priority: normal - low versions: -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Ezio Melotti added the comment: FWIW I'm +1 about adding grouper(), since it happened to me to use it and suggest more often than any other recipe (I think it's the only recipe I used/know), and even more often than some of the current itertools. The fact that has been requested several times already it's a good indication that it's something useful, and not a feature creep (I don't think any other recipe received nearly as many requests). Regarding use cases, a few days ago I needed it while shuffling a sequence of 20 chars and then split it in four 5-chars long groups. I also remember writing code like this more than once: s = 'abcd' n = 5 [s[x:x+n] for x in range(0,len(s),n)] ['a', 'b', 'c', 'd'] (As a side note, I find the grouper(iterable, n, fill) signature more intuitive than grouper(n, iterable, fill) (the recipe uses the latter).) -- assignee: rhettinger - nosy: +ezio.melotti stage: - needs patch type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Changes by Ezio Melotti ezio.melo...@gmail.com: -- assignee: - rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Terry J. Reedy added the comment: [Anatoly, 'Versions 3.5' is for changes that should *not* happen in 3.4, such as a planned removal for something deprecated in 3.3.] -- nosy: +terry.reedy versions: -Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: - rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17862 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17862] itertools.chunks(iterable, size, fill=None)
New submission from anatoly techtonik:
The history:
2007 - http://bugs.python.org/issue1502
2009 - http://bugs.python.org/issue6021
I'd like to resurrect this proposal again, but name it:
itertools.chunks(iterable, size, fill=None)
Two reasons.
1. practicality - top itertools request on StackOverflow
http://stackoverflow.com/search?tab=votesq=%5bpython%5d%20%2bitertools
2. performance
the time should be a constant for a fixed-length iterable regardless of a size
of chunk, but in fact the time is proportional to the size of chunk
{{{
import timeit
print timeit.timeit(
'grouper(3, x*40)', setup='from __main__ import grouper',
number=1000
)
print timeit.timeit(
'grouper(30, x*40)', setup='from __main__ import grouper',
number=1000
)
}}}
1.52581005407
14.6219704599
Addressing odd length user stories from msg87745:
1. no exceptions - odd end is an easy check if you need it
2. fill-in value - provided
3. truncation - just don't set fill-in value
3.1. if you really need fill-in as None, then an itertools.TRUNCATE value can
be used as a truncation parameter
4. partially filled-in tuple - not sure what that means
Raymond, your opinion is critical here. =)
--
components: Library (Lib)
messages: 187995
nosy: rhettinger, techtonik
priority: normal
severity: normal
status: open
title: itertools.chunks(iterable, size, fill=None)
versions: Python 3.3, Python 3.4, Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17862
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
