Serhiy Storchaka added the comment: Original Raymonds reason in msg248662 is not valid. Pickling a cycle object that fully consumed its input iterable is already space-inefficient.
>>> import itertools, pickle, pickletools >>> c = itertools.cycle(iter('abcde')) >>> [next(c) for i in range(8)] ['a', 'b', 'c', 'd', 'e', 'a', 'b', 'c'] >>> pickle.dumps(c) b'\x80\x03citertools\ncycle\nq\x00cbuiltins\niter\nq\x01]q\x02(X\x01\x00\x00\x00aq\x03X\x01\x00\x00\x00bq\x04X\x01\x00\x00\x00cq\x05X\x01\x00\x00\x00dq\x06X\x01\x00\x00\x00eq\x07e\x85q\x08Rq\tK\x03b\x85q\nRq\x0bh\x02K\x01\x86q\x0cb.' >>> pickletools.dis(pickle.dumps(c)) 0: \x80 PROTO 3 2: c GLOBAL 'itertools cycle' 19: q BINPUT 0 21: c GLOBAL 'builtins iter' 36: q BINPUT 1 38: ] EMPTY_LIST 39: q BINPUT 2 41: ( MARK 42: X BINUNICODE 'a' 48: q BINPUT 3 50: X BINUNICODE 'b' 56: q BINPUT 4 58: X BINUNICODE 'c' 64: q BINPUT 5 66: X BINUNICODE 'd' 72: q BINPUT 6 74: X BINUNICODE 'e' 80: q BINPUT 7 82: e APPENDS (MARK at 41) 83: \x85 TUPLE1 84: q BINPUT 8 86: R REDUCE 87: q BINPUT 9 89: K BININT1 3 91: b BUILD 92: \x85 TUPLE1 93: q BINPUT 10 95: R REDUCE 96: q BINPUT 11 98: h BINGET 2 100: K BININT1 1 102: \x86 TUPLE2 103: q BINPUT 12 105: b BUILD 106: . STOP highest protocol among opcodes = 2 An internal iterator is not pickled as iter("de"), but as an iterator of the list ["a", "b", "c", "d", "e"] with 3 items consumed. This list also saved as a part of a cycle object state, but not as a copy, but as a reference. There are two alternative patches. Both keep Raymonds optimization of cycle iterating, but have advantages. cycle_reduce_2.patch makes __reduce__ faster and more memory efficient than Raymonds variant. cycle_reduce_3.patch makes unpickled cycle object so optimized as original. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24874> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com