[issue44861] csv.writer stopped to quote values with escapechar with csv.QUOTE_MINIMAL in Python 3.10
Sebastian Bank added the comment: The 3.9 behaviour is write: "spam\eggs" The 3.10 behaviour is write: spam\\eggs I think at least the change in csv.QUOTE_MINIMAL behviour should be documented (maybe adding hint to avoid the `escapechar` option for consistent output). -- ___ Python tracker <https://bugs.python.org/issue44861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44861] csv.writer stopped to quote values with escapechar with csv.QUOTE_MINIMAL in Python 3.10
Sebastian Bank added the comment: IIUC there is no way to work around this from client/downstream code (to get the olf 3.6 to 3.9 behaviour), so this might break assertions on the output of `csv.writer` for users of `escapechar` whenever the data to be written contains the escapcechar (e.g. calculating a hash/checksum). -- ___ Python tracker <https://bugs.python.org/issue44861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44861] csv.writer stopped to quote values with escapechar with csv.QUOTE_MINIMAL in Python 3.10
New submission from Sebastian Bank :
AFAICT there was an undocumented change in behaviour related to the fix of
https://bugs.python.org/issue12178 (also reported in
https://bugs.python.org/issue12178#msg397440):
Python 3.9 quotes values with escapechar:
```
import csv
import io
kwargs = {'escapechar': '\\'}
value = 'spam\\eggs'
print(value)
with io.StringIO() as buf:
writer = csv.writer(buf, **kwargs)
writer.writerow([value])
line = buf.getvalue()
print(line.strip())
with io.StringIO(line) as buf:
reader = csv.reader(buf, **kwargs)
(new_value,), = reader
print(new_value)
spam\eggs
"spam\eggs"
spameggs
```
- quotes escapechar
- fails to double the escapechar (https://bugs.python.org/issue12178)
>From https://docs.python.org/3/library/csv.html#csv.QUOTE_MINIMAL
> only quote those fields which contain special characters
> such as delimiter, quotechar or any of the characters in
> lineterminator.
The previous behaviour seems incorrect because escapechar is not explicitly
mentioned, but at the same time the docs says 'such as'.
The new might better matching the name 'minimal', but at the same time one
might regard 'quote when in doubt' as a safer behaviour for the default quoting
rule.
Python 3.10:
https://github.com/python/cpython/blob/5c0eed7375fdd791cc5e19ceabfab4170ad44062/Lib/test/test_csv.py#L207-L208
See also https://github.com/xflr6/csv23/actions/runs/1027687524
--
components: Library (Lib)
messages: 399188
nosy: ebreck, taleinat, xflr6
priority: normal
severity: normal
status: open
title: csv.writer stopped to quote values with escapechar with
csv.QUOTE_MINIMAL in Python 3.10
type: behavior
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue44861>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12178] csv writer doesn't escape escapechar
Sebastian Bank added the comment:
Thanks Tal.
AFAICT there was an undocumented change in behaviour related to this fix.
Python 3.9 quotes values with escapechar:
```
import csv
import io
kwargs = {'escapechar': '\\'}
value = 'spam\\eggs'
print(value)
with io.StringIO() as buf:
writer = csv.writer(buf, **kwargs)
writer.writerow([value])
line = buf.getvalue()
print(line.strip())
with io.StringIO(line) as buf:
reader = csv.reader(buf, **kwargs)
(new_value,), = reader
print(new_value)
spam\eggs
"spam\eggs"
spameggs
```
- quotes escapechar
- fails to double the escapechar (this bug)
Btw, from
https://docs.python.org/3/library/csv.html#csv.QUOTE_MINIMAL
> only quote those fields which contain special characters
> such as delimiter, quotechar or any of the characters in lineterminator.
this seems incorrect because escapechar is not mentioned (but at the same time
it says 'such as') and maybe better matching the name 'minimal' (or one might
expect 'more' quoting as a better default).
Python 3.10:
https://github.com/python/cpython/blob/5c0eed7375fdd791cc5e19ceabfab4170ad44062/Lib/test/test_csv.py#L207-L208
See also https://github.com/xflr6/csv23/actions/runs/1027687524
--
___
Python tracker
<https://bugs.python.org/issue12178>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37549] os.dup() fails for standard streams on Windows 7
Change by Sebastian Bank : -- nosy: +xflr6 ___ Python tracker <https://bugs.python.org/issue37549> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34044] subprocess: reusing STARTUPINFO breaks under 3.7 (Windows)
Sebastian Bank added the comment: Perfect, thanks for the quick fix. -- ___ Python tracker <https://bugs.python.org/issue34044> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista
Sebastian Bank added the comment: Thanks Eryk. Done: https://bugs.python.org/issue34044 -- ___ Python tracker <https://bugs.python.org/issue19764> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34044] subprocess: reusing STARTUPINFO breaks under 3.7 (Windows)
New submission from Sebastian Bank : AFAIU, the change for https://bugs.python.org/issue19764 broke the following usage of subprocess on Windows (re-using a subprocess.STARTUPINFO instance to hide the command window): import os, subprocess STARTUPINFO = subprocess.STARTUPINFO() STARTUPINFO.dwFlags |= subprocess.STARTF_USESHOWWINDOW STARTUPINFO.wShowWindow = subprocess.SW_HIDE # raises OSError: [WinError 87] # in the second loop iteration starting with Python 3.7 for i in range(2): print(i) with open(os.devnull, 'w') as stderr: subprocess.check_call(['attrib'], stderr=stderr, startupinfo=STARTUPINFO) AFAICT, this works on Python 2.7, 3.4, 3.5, and 3.6 I think the documentation in https://docs.python.org/3/library/subprocess.html#windows-popen-helpers does not mention that every Popen call should be done with a fresh instance, so either the documentation needs to be changed, or the implementation (e.g. by deep-copying the instance). See also https://bugs.python.org/issue19764#msg320784 -- components: Library (Lib) messages: 321047 nosy: eryksun, vstinner, xflr6 priority: normal severity: normal status: open title: subprocess: reusing STARTUPINFO breaks under 3.7 (Windows) type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue34044> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista
Sebastian Bank added the comment: AFAIU, this change broke the following usage of subprocess on Windows (re-using a subprocess.STARTUPINFO instance to hide the command window): import os, subprocess STARTUPINFO = subprocess.STARTUPINFO() STARTUPINFO.dwFlags |= subprocess.STARTF_USESHOWWINDOW STARTUPINFO.wShowWindow = subprocess.SW_HIDE # raises OSError: [WinError 87] # in the second loop iteration starting with Python 3.7 for i in range(2): print(i) with open(os.devnull, 'w') as stderr: subprocess.check_call(['attrib'], stderr=stderr, startupinfo=STARTUPINFO) AFAICT, this works on Python 2.7, 3.4, 3.5, and 3.6 -- nosy: +xflr6 ___ Python tracker <https://bugs.python.org/issue19764> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE
Sebastian Bank <sebastian.b...@uni-leipzig.de> added the comment: To be complete, the docs of Dialect.escapechar should probably also say that it is used to escape itself. However, note that csw.writer currently only does this with csv.QUOTE_NONE (breaking round-trip otherwise: #12178). -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue15927> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12178] csv writer doesn't escape escapechar
Sebastian Bank <sebastian.b...@uni-leipzig.de> added the comment: Hi, is there something we can do to get this going? As the issue breaks round-trip, it currently requires work-arounds like this: https://github.com/cldf/csvw/blob/1324550266c821ef32d1e79c124191e93aefbfa8/csvw/dsv.py#L67-L71 -- nosy: +xflr6 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12178> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31590] CSV module incorrectly treats escaped newlines as new records if unquoted
Sebastian Bank <sebastian.b...@uni-leipzig.de> added the comment: https://bugs.python.org/issue15927#msg309811 gives sme code examples illustrating why I think this should be backported (and also the documentation should be changed for both Python 2 and 3). -- nosy: +xflr6 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31590> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15927] csv.reader() does not support escaped newline when quoting=csv.QUOTE_NONE
Sebastian Bank <sebastian.b...@uni-leipzig.de> added the comment:
I am not sure about the design vs. code bug distinction, but what makes me
think this should be fixed is primarily the broken round-trip (already
mentioned above):
>>> import io, csv
>>> def roundtrip(value, **fmtparams):
with io.BytesIO() as f:
csv.writer(f, **fmtparams).writerow([value])
f.seek(0)
return next(csv.reader(f, **fmtparams))
>>> roundtrip('spam\neggs', quoting=csv.QUOTE_NONE, escapechar='\\')
['spam\n']
Furthermore, there is the inconsistency between Python 2 and 3, now that this
has been fixed in 3.4.
I agree that the documentation of Dialect.escapechar is not in line with the
code (in both Python 2 and Python 3): How about changing it to something along
the following lines (TODO: reformulate according to how exactly
Dialect.lineterminator affects this)?
"to escape the delimiter, \r, \n, and the quotechar if quoting is set to
QUOTE_NONE
and the quotechar for all other quoting styles if doublequote is False":
>>> def write_csv(value, **fmtparams):
with io.BytesIO() as f:
csv.writer(f, **fmtparams).writerow([value])
return f.getvalue()
>>> write_csv('spam\reggs', quoting=csv.QUOTE_NONE, escapechar='\\')
'spam\\\reggs\r\n'
>>> write_csv('spam\neggs', quoting=csv.QUOTE_NONE, escapechar='\\')
'spam\\\neggs\r\n'
>>> write_csv('spam"eggs', quoting=csv.QUOTE_NONE, escapechar='\\')
'spam\\"eggs\r\n'
>>> write_csv('spam"eggs', quoting=csv.QUOTE_NONE, quotechar=None,
>>> escapechar='\\')
'spam"eggs\r\n'
>>> write_csv('spam"eggs', escapechar='\\', doublequote=False)
'spam\\"eggs\r\n'
> In any case, 'one\nelement' and 'one\\\nelement' are each 2 physical lines.
> I don't see anything in the doc about csv.reader joining physical lines
> into 'logical' lines the way that compile() does.
How about the following?
"csvreader.line_num
The number of lines read from the source iterator. This is not the same as
the number of records returned, as records can span multiple lines."
"On reading, the escapechar removes any special meaning from the following
character."
>>> write_csv('spam\neggs', quoting=csv.QUOTE_NONE) # with delimiter, \r, \n,
>>> and quotechar
Traceback (most recent call last):
...
Error: need to escape, but no escapechar set
>>> roundtrip('spam\neggs')
['spam\neggs']
>>> write_csv('spam\neggs')
'"spam\neggs"\r\n'
--
nosy: +xflr6
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue15927>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26327] File > Save in IDLE shell window not working
New submission from Sebastian Bank: Under Python 2.7.11 (Win 7), saving of the IDLE shell output produces no file if the output contains non-ASCII characters, e.g. after doing (before this, it does work): >>> print u'sp\xe4m' späm >>> When saving (generally), the cursor also moves to the next line. Maybe the default file type for the shell save dialog(s) can be changed from 'Python Files (*.py, *.pyw)' to the other entry 'Text files (*.txt)' as the resulting file will normally not be a valid Python file (e.g. due to '>>>' prompts). -- components: IDLE messages: 260008 nosy: xflr6 priority: normal severity: normal status: open title: File > Save in IDLE shell window not working type: behavior versions: Python 2.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26327> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20923] ConfigParser should nested [] in section names.
Sebastian Bank added the comment: Terry: I am not so sure about that interpretation. Do we agree that the INI-files are the data/message? ConfigParser refuses to accept dirty INI-Files (with ']' in section names) but will produce this kind of files. I we see the arguments given to ConfigParser as data/message, it does indeed accept dirty data as you say, but still it does not emit clean one in that case, right? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20923> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20923] ConfigParser should nested [] in section names.
Sebastian Bank added the comment: If this is the intended behaviuour, I guess ConfigParser should warn the user if he supplies a section name with a ']' (prevent failed roundtrips). See http://bugs.python.org/issue23301 The current behaviour looks like the opposite of Postel's law. -- nosy: +xflr6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20923 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23301] ConfigParser does not handle square brackets in section name
New submission from Sebastian Bank:
ConfigParser parses section lines containing square brackets like '[spam [eggs]
spam]' up to the first instead of the last occurrence of ']' preventing
roundtrips:
s = StringIO()
c1 = ConfigParser()
c1.add_section('spam [eggs]')
c1.write(s)
s.seek(0)
c2 = ConfigParser()
c2.readfp(s)
assert c1.sections() == c2.sections() # fails
Potential fix: change the second line of SECTCRE from r'(?Pheader[^]]+)' to
r'(?Pheader.+?)'.
If the parsing behaviour cannot be changed, the user should at least be warned
about supplying data that breaks the roundtrip.
--
components: Library (Lib)
messages: 234497
nosy: xflr6
priority: normal
severity: normal
status: open
title: ConfigParser does not handle square brackets in section name
type: behavior
versions: Python 2.7, Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23301
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
