[issue23041] csv needs more quoting rules

Samwyse added the comment: I just signed the contributor agreement. (Thought I had done that last year but I don’t see any emails. Is there any place to check?) I agree that round-tripping should Bebe possible for any value of quoting. Hopefully this will finally get done before its eighth

[issue23041] csv needs more quoting rules

Miha Šetina added the comment: Is this still on track for python 3.11? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue23041] csv needs more quoting rules

Serhiy Storchaka added the comment: The quoting style affects not only the CSV formatting, but also the CSV parsing. How QUOTE_NOTNULL and QUOTE_STRINGS will affect parsing? -- ___ Python tracker

[issue23041] csv needs more quoting rules

Miha Šetina added the comment: I would say that it should follow a rule: both cases values of None are output as an empty field so empty filed should map to None with both QUOTE_NOTNULL and QUOTE_STRINGS that would make: ["", None, 1, "a"] for QUOTE_STRINGS in effect. This would also make

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Further question... All the discussion has been on the writer side of the csv module. Is there any reason that using QUOTE_STRINGS or QUOTE_NOTNULL should have an effect when reading? For example, should this line on input "",,1,'a' produce this list ["",

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Note to @samwyse and @krypten: I updated the patches and created a pull request on GitHub, but I have no way of knowing if at least krypten has signed a CLA for Python. Since you're the author of the original patches, we need to verify that you have.

[issue23041] csv needs more quoting rules

Change by Skip Montanaro : -- pull_requests: +27722 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/29469 ___ Python tracker ___

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Update version - too late for anything older than 3.11. -- versions: -Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker

[issue23041] csv needs more quoting rules

Miha Šetina added the comment: The MS Synapse has a CSV support in its COPY command that would benefit from the proposed csv.QUOTE_NOTNULL as it can be used when preparing data for import. As they reference the same RFC the CSV modul should support, both the proposed modifications would

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Ugh... s/QUOTE_NONNULL/QUOTE_NOTNULL/ Not, Non, None... Perl would treat them all the same, right? -- ___ Python tracker ___

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Missed tweaking a couple settings. -- resolution: rejected -> stage: resolved -> needs patch versions: +Python 3.11 -Python 3.8 ___ Python tracker

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Okay, I'll reopen this, at least for the discussion of QUOTE_NONNULL. @erdnaxeli please given an example of how PostgreSQL distinguishes between the empty string and None cases. Is it a quoted empty string vs an empty field? If so, modifying @samwyse's

[issue23041] csv needs more quoting rules

Alexandre Morignot added the comment: I have another use case. I want to import data into PostgreSQL using the COPY FROM command. This command can read a CSV input and it needs to distinguish empty string from null values. Could we reconsider this issue and the proposed solution? --

[issue23041] csv needs more quoting rules

Yoong Hor Meng added the comment: There is a real requirement for csv to handle an empty field vs a empty string . csv.QUOTE_NOTNULL could be useful. -- nosy: +yoonghm versions: +Python 3.8 -Python 3.6 ___ Python tracker

[issue23041] csv needs more quoting rules

tegdev added the comment: The correct handling of None values belongs to the csv module. There is a use case to migrate a DB2 database to PostgreSQL. DB2 has a command line tool "db2 export ..." which produces csv-files. A row ['Hello', null, 'world'] is exported to "Hello,,"world". I

[issue23041] csv needs more quoting rules

Berker Peksag added the comment: I was thinking adding a more flexible API like: ... spamwriter = csv.writer(csvfile, quoting_callable=lambda field: field is not None) ... But that would require too much change in the csv module (or at least its implementation wouldn't be

[issue23041] csv needs more quoting rules

Serhiy Storchaka added the comment: The csv module is already type-sensitive (with QUOTE_NONNUMERIC). I agree, that we shouldn't modify the csv module just for one user and one program. If a standard CVS library in Java (or other popular laguages) differentiates between empty string and null

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: Thanks for the update berker.peksag. I'm still not convinced that the csv module should be modified just so one user (sorry samwyse) can match the input format of someone's Java program. It seems a bit like trying to make the csv module type-sensitive. What

[issue23041] csv needs more quoting rules

Changes by Berker Peksag : -- nosy: +berker.peksag stage: needs patch -> patch review versions: +Python 3.6 -Python 3.5 ___ Python tracker

[issue23041] csv needs more quoting rules

Samwyse added the comment: Yes, it's based on a real-world need. I work for a Fortune 500 company and we have an internal tool that exports CSV files using what I've described as the QUOTE_NOTNULL rules. I need to create similar files for re-importation. Right now, I have to post-process

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: If I understand correctly, your software needs to distinguish between # wrote [foo, , 42, None] with quote_all in effect foo,,42, and # wrote [foo, None, 42, ] with quote_nonnull in effect foo,,42, so you in effect want to transmit some type information

[issue23041] csv needs more quoting rules

Samwyse added the comment: Skip, I don't have any visibility into how the Java program I'm feeding data into works, I'm just trying to replicate the csv files that it exports as accurately as possible. It has several other quirks, but I can replicate all of them using Dialects; this is the

[issue23041] csv needs more quoting rules

Skip Montanaro added the comment: It doesn't look like a difficult change, but is it really needed? I guess my reaction is the same as Raymond's. Are there real-world uses where the current set of quoting styles isn't sufficient? -- ___ Python

[issue23041] csv needs more quoting rules

Chaitanya agrawal added the comment: Used function PyUnicode_Check instead of PyString_Check -- keywords: +patch nosy: +krypten Added file: http://bugs.python.org/file37444/issue23041.patch ___ Python tracker rep...@bugs.python.org

[issue23041] csv needs more quoting rules

Changes by Chaitanya agrawal chaitiagra...@gmail.com: Added file: http://bugs.python.org/file37445/issue23041_test.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23041 ___

[issue23041] csv needs more quoting rules

New submission from Samwyse: The csv module currently implements four quoting rules for dialects: QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC and QUOTE_NONE. These rules treat values of None the same as an empty string, i.e. by outputting two consecutive quotes. I propose the addition of two

[issue23041] csv needs more quoting rules

R. David Murray added the comment: As an enhancement, this could be added only to 3.5. The proposal sounds reasonable to me. -- keywords: +easy nosy: +r.david.murray stage: - needs patch versions: -Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.6

[issue23041] csv needs more quoting rules

Samwyse added the comment: David: That's not a problem for me. Sorry I can't provide real patches, but I'm not in a position to compile (much less test) the C implementation of _csv. I've looked at the code online and below are the changes that I think need to be made. My use cases don't

[issue23041] csv needs more quoting rules

Raymond Hettinger added the comment: Samwyse, are these suggestions just based on ideas of what could be done or have you encountered real-world CSV data exchanges that couldn't be handled by the CSV module? -- assignee: - skip.montanaro nosy: +rhettinger, skip.montanaro