[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Serhiy Storchaka added the comment: This issue is outdated since implementing automatic encoding detecting in issue17909. -- resolution: -> out of date stage: -> resolved status: open -> closed ___ Python tracker

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Santoso Wijaya added the comment: I think you should use codecs.BOM_UTF8 rather than using hardcoded string \xef\xbb\xbf directly. And why special casing UTF-8 while we're at it? What about other encodings and their BOMs? -- nosy: +santa4nt ___

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Kristian Benoit added the comment: I added code to skip the bom if present when encoding is either None or utf-8. The problem I have with Victor's solution is that users don't know these files are not plain UTF-8. Most text editor says it's utf-8 encoded, how can a user figure out there 3

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Changes by Kristian Benoit kristian.ben...@gmail.com: Added file: http://bugs.python.org/file35270/json.v2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21509 ___

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Serhiy Storchaka added the comment: Currently json.load/loads don't support binary input. See issue17909 and issue19837. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21509 ___

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Chris Rebert added the comment: The new JSON RFC now at least mentions BOM handling: https://tools.ietf.org/html/rfc7159#section-8.1 : Implementations MUST NOT add a byte order mark to the beginning of a JSON text. In the interests of interoperability, implementations that parse JSON texts

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

New submission from Kristian Benoit: I'm trying to parse a json and keep getting ValueError. File reports the file as being UTF-8 Unicode (with BOM) text, vim reports it as UTF-8, ... json.load docs says it support UTF-8 out of the box. Here's a link to the file :

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

STINNER Victor added the comment: In Python 2, json.loads() accepts str and unicode types. You can support JSON starting with a UTF-8 BOM using the Python codec utf-8-sig. Example: codecs.BOM_UTF8 + b'{\n}' '\xef\xbb\xbf{\n}' json.loads(codecs.BOM_UTF8 + b'{\n}') Traceback (most recent call

[issue21509] json.load fails to read UTF-8 file with (BOM) Byte Order Marks

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21509 ___ ___ Python-bugs-list