[issue4847] csv fails when file is opened in binary mode

R. David Murray added the comment: Doc patch applied in r71116. -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___

[issue4847] csv fails when file is opened in binary mode

R. David Murray added the comment: Oh, yeah, other 3.1 differences are that the unicode test is uncommented and updated, and a test is added to make sure nulls are handled correctly. -- ___ Python tracker _

[issue4847] csv fails when file is opened in binary mode

R. David Murray added the comment: > Is this about correct? Do any test cases need to be updated or added? I > notice that something called BytesIO is imported from io but not used. Were > some test cases removed which used to involve that class or is that a 2to3 > artifact? The set of test

[issue4847] csv fails when file is opened in binary mode

R. David Murray added the comment: Nope. Sorry I forgot to change the priority. -- priority: release blocker -> normal ___ Python tracker ___ ___

[issue4847] csv fails when file is opened in binary mode

Benjamin Peterson added the comment: So this is a doc bug? If so, need it still block tomorrow's release? -- nosy: +benjamin.peterson ___ Python tracker ___ _

[issue4847] csv fails when file is opened in binary mode

R David Murray added the comment: On Thu, 2 Apr 2009 at 17:55, Skip Montanaro wrote: > This thread is getting a bit long. Can someone summarize how the expected > usage of the csv module is supposed to change? If I read things correctly, > instead of requiring (in the general case) that csv fi

[issue4847] csv fails when file is opened in binary mode

Skip Montanaro added the comment: David> I also deleted the unicode discussion (since CSV obviously David> handles unicode now) ... Maybe there should be a simple example showing use of the encoding parameter to open() to encode Unicode on write and decode to Unicode on read? Skip ---

[issue4847] csv fails when file is opened in binary mode

Skip Montanaro added the comment: David> I've added some unit tests for embedded newlines, and py3k csv David> passes (on linux at least) when newline='' is used. Unless David> someone can provide a test case that fails when newline='' is David> used, I propose we fix the documentat

[issue4847] csv fails when file is opened in binary mode

Changes by R. David Murray : -- assignee: georg.brandl -> r.david.murray Added file: http://bugs.python.org/file13570/issue4847-doc.patch ___ Python tracker ___ __

[issue4847] csv fails when file is opened in binary mode

Changes by R. David Murray : Removed file: http://bugs.python.org/file13569/issue4847-doc.patch ___ Python tracker ___ ___ Python-bugs-list mai

[issue4847] csv fails when file is opened in binary mode

R. David Murray added the comment: I'm attaching a proposed doc patch for comment. I replace mentions of 'rb' with "newline=''", including in the examples. I also deleted the unicode discussion (since CSV obviously handles unicode now) as well as the extensive unicode examples that are no long

[issue4847] csv fails when file is opened in binary mode

R. David Murray added the comment: I've added some unit tests for embedded newlines, and py3k csv passes (on linux at least) when newline='' is used. Unless someone can provide a test case that fails when newline='' is used, I propose we fix the documentation and leave the code alone.

[issue4847] csv fails when file is opened in binary mode

Guido van Rossum added the comment: I think it's good if it allowed passing in a binary file and an encoding, but I think it would be crazy if it wouldn't also take a text file. After all the primary purpose of a CSV file, edge cases notwithstanding, is to look like text to the end user in a te

[issue4847] csv fails when file is opened in binary mode

Skip Montanaro added the comment: This issue seems to have simply been overlooked when 3.0 was released. It should be fixed in the next round of 3.0 and 3.1 updates. Any feeback on the idea that the csv.reader constructor (and probably the DictReader and proposed NamedTupleReader constructors)

[issue4847] csv fails when file is opened in binary mode

Skip Montanaro added the comment: me> What should be the default? Scratch that. If the iterator passed to csv.reader is in a mode which will cause it to emit bytes instead of unicode objects the caller must give an encoding. The csv.reader code will then perform the necessary bytes-to-unicode

[issue4847] csv fails when file is opened in binary mode

Antoine Pitrou added the comment: > I had already read it. I gave it a prize for "least intuitive arg in the > language". Please open a bug, then :) > So you plan to use that, reading "lines" instead of blocks? > You'll still have to examine which CRs and LFs are embedded and which > are line

[issue4847] csv fails when file is opened in binary mode

John Machin added the comment: pitrou> Please look at the doc for open() and io.TextIOWrapper. The `newline` parameter defaults to None, which means universal newlines with newline translation. Setting to '' (yes, the empty string) enables universal newlines but disables newline translation ...

[issue4847] csv fails when file is opened in binary mode

Antoine Pitrou added the comment: > Not good. It should preserve ALL characters in the field. Please look at the doc for open() and io.TextIOWrapper. The `newline` parameter defaults to None, which means universal newlines with newline translation. Setting to '' (yes, the empty string) enables

[issue4847] csv fails when file is opened in binary mode

John Machin added the comment: ... and it looks like Option 2 might already *almost* be in place. Continuing with the previous example (book1.csv has embedded lone LFs): C:\devel\csv>\python30\python -c "import csv; print(repr(list(csv.reader(open('book1.csv','rt', encoding='ascii')" [['Fie

[issue4847] csv fails when file is opened in binary mode

John Machin added the comment: Before patching, could we discuss the requirements? There are two different concepts: (1) "text" file (assume that CR and/or LF are line terminators, and provide methods for accessing a line at a time) versus "binary" file (no such assumptions, no such access) (2)

[issue4847] csv fails when file is opened in binary mode

Jervis Whitley added the comment: Hi all, This patch takes the approach of assuming utf-8 format encoding for files opened with 'rb' directive. That is: 1. Check if each line is Unicode Or Bytes Type. 2. If Bytes, get char array reference to internal buffer. 3. use PyUnicode_FromString to cr

[issue4847] csv fails when file is opened in binary mode

John Machin added the comment: Sorry, folks, we've got an understanding problem here. CSV files are typically NOT created by text editors. They are created e.g. by "save as csv" from a spreadsheet program, or as an output option by some database query program. They can have just about any charac

[issue4847] csv fails when file is opened in binary mode

STINNER Victor added the comment: Short patch fixing the examples + the description of reader() and writer() (remove the "b" flag sentence: it's wrong, we need unicode!). I hope that I didn't break the documentation (i tried "make text" and i didn't get any warning). -- keywords: +pa

[issue4847] csv fails when file is opened in binary mode

jaywalker added the comment: I think what you suggest makes most sense. Thanks. - Original Message From: Antoine Pitrou To: jaywal...@yahoo.com Sent: Monday, January 5, 2009 12:47:21 PM Subject: [issue4847] csv fails when file is opened in binary mode Antoine Pitrou added the

[issue4847] csv fails when file is opened in binary mode

Antoine Pitrou added the comment: You can avoid the newline translation problem by using the newline parameter in open(). Set it to '' (the empty string) and any CR and LF characters should remain intact. As for the original problem, IMHO it is a documentation bug. -- assignee: -> geo

[issue4847] csv fails when file is opened in binary mode

s, \r gets lost. Not a biggie, but you never know ;0) - Original Message From: STINNER Victor To: jaywal...@yahoo.com Sent: Monday, January 5, 2009 12:23:45 PM Subject: [issue4847] csv fails when file is opened in binary mode STINNER Victor added the comment: > say one of the

[issue4847] csv fails when file is opened in binary mode

STINNER Victor added the comment: > say one of the fields has an embedded \r. For instance "blahblah\r" is the > value of the first column. Now open this file in text mode. What happens to > this '\r' even before csv.reader sees it? I used rarely the CSV format, but it sounds strange to have a

[issue4847] csv fails when file is opened in binary mode

o problem. If it is converted to a newline in any platform, and that newline is not exactly \r, does this not introduce a problem? Just curious... Thanks. - Original Message From: STINNER Victor To: jaywal...@yahoo.com Sent: Monday, January 5, 2009 12:05:59 PM Subject: [issue4847] cs

[issue4847] csv fails when file is opened in binary mode

STINNER Victor added the comment: Do you expect to be able to read CSV as bytes or just to fix the documentation example? -- nosy: +haypo ___ Python tracker ___

[issue4847] csv fails when file is opened in binary mode

New submission from jaywalker : The following code from the documentation fails: # import csv reader = csv.reader(open("eggs.csv", "rb")) for row in reader: print(row) # The output is: Traceback (most recent call last): File "", line 1, in File "", lin