[issue17343] Add a version of str.split which returns an iterator

Juancarlo Añez added the comment: def isplit(text, sep=None, maxsplit=-1): """ A lowmemory-footprint version of: iter(text.split(sep, maxsplit)) Adapted from https://stackoverflow.com/a/9770397 """ if maxsplit == 0: yield text else: rsep = re.es

[issue17343] Add a version of str.split which returns an iterator

Paweł Miech added the comment: Making string.split iterator sounds like an interesting task. I found this issue because recently we talked in project that string.split returns a list and it can cause increased memory usage footprint for some tasks when there is large response to parse. Here

[issue17343] Add a version of str.split which returns an iterator

Change by Brett Cannon : -- nosy: -brett.cannon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue17343] Add a version of str.split which returns an iterator

Martin Winks added the comment: > Perhaps the use case is already served by re.finditer() def split_whitespace_ascii(s: str): return (pt.group(0) for pt in re.finditer(r"[A-Za-z']+", s)) solution above does not cover all possible data and is incorrect for bytes-like objects. writing reg

[issue17343] Add a version of str.split which returns an iterator

Changes by Serhiy Storchaka : -- resolution: -> rejected stage: needs patch -> resolved status: pending -> closed ___ Python tracker ___

[issue17343] Add a version of str.split which returns an iterator

Changes by Serhiy Storchaka : -- status: open -> pending ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:

[issue17343] Add a version of str.split which returns an iterator

Raymond Hettinger added the comment: No one has submitted a patch for this or has expressed an interest in a long time. Perhaps the use case is already served by re.finditer() Unassigning. Feel free to push this forward or to close due to lack on interest. -- assignee: rhettinger ->

[issue17343] Add a version of str.split which returns an iterator

Changes by Giampaolo Rodola' : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue17343] Add a version of str.split which returns an iterator

Terry J. Reedy added the comment: > I'm guessing Terry wanted to say "os.listdir" instead of "os.walk". yes, sorry. -- ___ Python tracker ___

[issue17343] Add a version of str.split which returns an iterator

Serhiy Storchaka added the comment: May be str.iter_indices() or even just str.indices()? -- ___ Python tracker ___ ___ Python-bugs-li

[issue17343] Add a version of str.split which returns an iterator

Georg Brandl added the comment: I'm guessing Terry wanted to say "os.listdir" instead of "os.walk". -- nosy: +georg.brandl ___ Python tracker ___

[issue17343] Add a version of str.split which returns an iterator

Raymond Hettinger added the comment: If someone wants whip-up a patch for str.iter_index(), I would be happy to review it. Be sure to add a test case to make sure that the results are non-overlapping: list(''.iter_index('aa')) == [0, 2] -- assignee: -> rhettinger

[issue17343] Add a version of str.split which returns an iterator

Raymond Hettinger added the comment: Alex, it was response to Terry's message: http://bugs.python.org/issue17343#msg183782 FWIW, I'm +1 on an iterator version of str.split(). I'm not sure yet that it would be worthwhile to propagate the idea to other string-like objects though. -- _

[issue17343] Add a version of str.split which returns an iterator

Alex Gaynor added the comment: Raymond: Is that for the wrong ticket, or was the message incorrect? :) -- ___ Python tracker ___ ___ P

[issue17343] Add a version of str.split which returns an iterator

Raymond Hettinger added the comment: -1 on os.walk returning an iterator. The API is already a bit challenging for some and our experience with itertools.groupby() is that returning an inner iterator can be very confusing. -- nosy: +rhettinger ___

[issue17343] Add a version of str.split which returns an iterator

Gregory P. Smith added the comment: It'd perhaps have been better if things like memoryview were never exposed to the user at all as a distinct type and became an internal implementation detail behind PyBytes and PyUnicode objects (they could hold a reference to something else or collapse that

[issue17343] Add a version of str.split which returns an iterator

Terry J. Reedy added the comment: I personally would have changed both str.split and os.walk to return iterators in 3.0, like many other builtins. The rationale for os.walk continuing to produce a list is that there would be little time saving as the list is not *that* long and most uses look

[issue17343] Add a version of str.split which returns an iterator

Changes by Tshepang Lekhonkhobe : -- nosy: +tshepang ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue17343] Add a version of str.split which returns an iterator

Serhiy Storchaka added the comment: > There is no string view that I know of. Interesting idea, though, thanks to > the immutability of strings. Would much have to be different other than > boundary checking and __hash__ (and hoping extension authors are changing > things in-place)? Objects/s

[issue17343] Add a version of str.split which returns an iterator

Serhiy Storchaka added the comment: > Indeed, a bytearray version would require the talked about but not > implemented due to complexity (in pep3118) support for locking a buffer from > other mutations. I rather think that a bytearray version can't pre-scan the data. Note that an array for pr

[issue17343] Add a version of str.split which returns an iterator

Brett Cannon added the comment: There is no string view that I know of. Interesting idea, though, thanks to the immutability of strings. Would much have to be different other than boundary checking and __hash__ (and hoping extension authors are changing things in-place)? I say go ahead and op

[issue17343] Add a version of str.split which returns an iterator

Gregory P. Smith added the comment: Indeed, a bytearray version would require the talked about but not implemented due to complexity (in pep3118) support for locking a buffer from other mutations. best concentrate on bytes then. Do we have a memoryview equivalent for PyUnicode? If not, we sh

[issue17343] Add a version of str.split which returns an iterator

Changes by Santoso Wijaya : -- nosy: +santa4nt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue17343] Add a version of str.split which returns an iterator

Changes by Brett Cannon : -- nosy: +brett.cannon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue17343] Add a version of str.split which returns an iterator

Serhiy Storchaka added the comment: > While not required, It'd be useful if the implementation of this pre-scanned > the data internally so that the length of the generated sequence was known up > front. This could imply an internal bitset of vector of split indices is > kept for the life of

[issue17343] Add a version of str.split which returns an iterator

Gregory P. Smith added the comment: The bytes (and bytearray?) version of this should generate memoryview's instead of new bytes objects to avoid a copy. While not required, It'd be useful if the implementation of this pre-scanned the data internally so that the length of the generated sequenc

[issue17343] Add a version of str.split which returns an iterator

New submission from Alex Gaynor: str.split returns a list, which is inefficient when you just want to process items one be one. You could emulate this with str.find and tracking indexes manually, but this should really be a builtin behavior. -- messages: 183411 nosy: alex priority: nor