[issue20528] fileinput module will read whole file into memory when using fileinput.hook_encoded due to codecs.StreamReader.readlines

2014-02-06 Thread R. David Murray
R. David Murray added the comment: Unfortunately, since 2.7 is in feature freeze, we can't add any new API there to allow a circumvention. On the other hand, it sounds like Serhiy has come up with an actual fix in the other issue, based on the reason that it is not in fact a problem in 3.x (t

[issue20528] fileinput module will read whole file into memory when using fileinput.hook_encoded due to codecs.StreamReader.readlines

2014-02-06 Thread Gunnar Aastrand Grimnes
New submission from Gunnar Aastrand Grimnes: When reading large files with fileinput, it will work as expected and only process a line at a time when used normally, but if you add an hook_encoded openhook it will read the whole file into memory before returning the first line. Verify by runn