[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Sebastian Bank]

Sebastian Bank  added the comment:

To be complete, the docs of Dialect.escapechar should probably also say that it 
is used to escape itself.
However, note that csw.writer currently only does this with csv.QUOTE_NONE 
(breaking round-trip otherwise: #12178).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Sebastian Bank]

Sebastian Bank  added the comment:

I am not sure about the design vs. code bug distinction, but what makes me 
think this should be fixed is primarily the broken round-trip (already 
mentioned above): 

>>> import io, csv
>>> def roundtrip(value, **fmtparams):
with io.BytesIO() as f:
 csv.writer(f, **fmtparams).writerow([value])
 f.seek(0)
 return next(csv.reader(f, **fmtparams))
>>> roundtrip('spam\neggs', quoting=csv.QUOTE_NONE, escapechar='\\')
['spam\n']

Furthermore, there is the inconsistency between Python 2 and 3, now that this 
has been fixed in 3.4.

I agree that the documentation of Dialect.escapechar is not in line with the 
code (in both Python 2 and Python 3): How about changing it to something along 
the following lines (TODO: reformulate according to how exactly 
Dialect.lineterminator affects this)?

"to escape the delimiter, \r, \n, and the quotechar if quoting is set to 
QUOTE_NONE
and the quotechar for all other quoting styles if doublequote is False":

>>> def write_csv(value, **fmtparams):
with io.BytesIO() as f:
csv.writer(f, **fmtparams).writerow([value])
return f.getvalue()
>>> write_csv('spam\reggs', quoting=csv.QUOTE_NONE, escapechar='\\')
'spam\\\reggs\r\n'
>>> write_csv('spam\neggs', quoting=csv.QUOTE_NONE, escapechar='\\')
'spam\\\neggs\r\n'
>>> write_csv('spam"eggs', quoting=csv.QUOTE_NONE, escapechar='\\')
'spam\\"eggs\r\n'
>>> write_csv('spam"eggs', quoting=csv.QUOTE_NONE, quotechar=None, 
>>> escapechar='\\')
'spam"eggs\r\n'
>>> write_csv('spam"eggs', escapechar='\\', doublequote=False)
'spam\\"eggs\r\n'

> In any case, 'one\nelement' and 'one\\\nelement' are each 2 physical lines. 
> I don't see anything in the doc about csv.reader joining physical lines
> into 'logical' lines the way that compile() does.

How about the following?

"csvreader.line_num

The number of lines read from the source iterator. This is not the same as 
the number of records returned, as records can span multiple lines."

"On reading, the escapechar removes any special meaning from the following 
character."

>>> write_csv('spam\neggs', quoting=csv.QUOTE_NONE)  # with delimiter, \r, \n, 
>>> and quotechar
Traceback (most recent call last):
...
Error: need to escape, but no escapechar set

>>> roundtrip('spam\neggs')
['spam\neggs']

>>> write_csv('spam\neggs')
'"spam\neggs"\r\n'

--
nosy: +xflr6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Terry J. Reedy]

Terry J. Reedy  added the comment:

This issue was 'reopened' by #31590.

I can understand inconsistency as a 'design bug', but design bugs are not code 
bugs, and fixing a design bugs is an enhancement issue, not a behavior issue.

It is not clear to me why, with the specified dialect,  
"writer.writerow(['one\nelement'])" is correct in writing 'one\\\nelement\n'.  
The doc for Dialect.excapechar says "A one-character string used by the writer 
to escape the delimiter if quoting is set to QUOTE_NONE and the quotechar if 
doublequote is False."  Yes, quoting is set to QUOTE_NONE, but \n is the 
lineterminator, not the delimiter (or the quotechar).  It looks to me that 
escaping the lineterminator might be a bug.

In any case, 'one\nelement' and 'one\\\nelement' are each 2 physical lines.  I 
don't see anything in the doc about csv.reader joining physical lines into 
'logical' lines the way that compile() does.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Michael Johnson]

Michael Johnson added the comment:

On input, the reader sees a line like 

['one\\\n','element']

from the file iterator and successfully escapes the newline character, but 
still interprets the end of the string as the end of a record.  I've attached a 
patch that modifies this behavior, so that encountering the end of a string 
immediately after an escaped \r or \n is does not begin a new record.

--
keywords: +patch
nosy: +mjohnson
Added file: http://bugs.python.org/file29495/csv.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Roundup Robot]

Roundup Robot added the comment:

New changeset 940748853712 by R David Murray in branch 'default':
#15927: Fix cvs.reader parsing of escaped \r\n with quoting off.
http://hg.python.org/cpython/rev/940748853712

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[R. David Murray]

R. David Murray added the comment:

Although this is clearly a bug fix, it also represents a behavior change that 
could cause a working program to fail.  I have therefore only applied it to 
3.4, but I'm open to arguments that it should be backported.

Thanks for the patch, Michael.

--
assignee: lukasz.langa - 
nosy: +r.david.murray
resolution:  - fixed
stage:  - committed/rejected
status: open - closed
versions:  -Python 2.7, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Kalon Mills]

Kalon Mills added the comment:

Serhiy,  sorry I'm not sure I understand your question.  But if you take a look 
at the script that exhibits the problem I think the bug that I'm reporting 
becomes more clear.

Namely, using the dialect configuration shown in the script, the round trip 
conversion from string through writer then through the reader back to string is 
inconsistent.  The reader should return as output the same input that was given 
to the corresponding writer and this is not the case.  

So even if CVS in not well defined I believe the writer and reader should at 
least be consistent.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

CSV is not well defined format. What you expect to read from csv.reader(['one', 
'two'])? If two rows ['one'] and ['two'], than the reader in its own right and 
there is no bug which can be fixed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Maciej Szulik]

Maciej Szulik added the comment:

I've confirmed that bug in the latest repo version, still exists. I attach 
patch for py3k. 
I'll try to have a look at it in the current version, as soon as it will be 
fixed I'll port it to 2.7.

--
nosy: +maciej.szulik
versions: +Python 3.4
Added file: http://bugs.python.org/file27509/test_csv_py3k.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Łukasz Langa]

Changes by Łukasz Langa luk...@langa.pl:


--
assignee:  - lukasz.langa
nosy: +lukasz.langa
versions: +Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

[Chris Jerdonek]

Changes by Chris Jerdonek chris.jerdo...@gmail.com:


--
components: +Library (Lib) -None
title: cvs.reader does not support escaped newline when quoting=cvs.QUOTE_NONE 
- csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15927
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com