[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16 or UTF-8)
Sebastian Ortiz Vasquez added the comment: I have been working with this in order to generate an RSS feed using web2py. I found, XMLGenerator method does not validate if is an unicode or string type, and it does not encode accord the encoding parameter of the XMLGenerator. I added changed the method to verify if is an unicode object or try to convert to it using the desired encoding. Recall that the _write UnbufferedTextIOWrapper receives an unicode object as parameter. def characters(self, content): if isinstance(content, unicode): self._write(escape(content)) else: self._write(escape(unicode(content,self._encoding))) -- nosy: +neoecos title: Bugfix for #1470540 (XMLGenerator cannot output UTF-16) - Bugfix for #1470540 (XMLGenerator cannot output UTF-16 or UTF-8) versions: -Python 3.2, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file29623/saxutils.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Roundup Robot added the comment: New changeset d707e3345a74 by Serhiy Storchaka in branch '2.7': Issue #1470548: Do not buffer XMLGenerator output. http://hg.python.org/cpython/rev/d707e3345a74 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Roundup Robot added the comment: New changeset 1c03e499cdc2 by Serhiy Storchaka in branch '3.2': Issue #1470548: Add test for fragment producing with XMLGenerator. http://hg.python.org/cpython/rev/1c03e499cdc2 New changeset 5a4b3094903f by Serhiy Storchaka in branch '3.3': Issue #1470548: Add test for fragment producing with XMLGenerator. http://hg.python.org/cpython/rev/5a4b3094903f New changeset 810d70fb17a2 by Serhiy Storchaka in branch 'default': Issue #1470548: Add test for fragment producing with XMLGenerator. http://hg.python.org/cpython/rev/810d70fb17a2 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka added the comment: Thank you for report. Here is a patch which fixes this bug. -- Added file: http://bugs.python.org/file29218/XMLGenerator_fragment-2.7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Arfrever Frehtes Taifersar Arahesis added the comment: This patch works for me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Arfrever Frehtes Taifersar Arahesis added the comment:
The change in 2.7 branch breaks some software, including a test of Django
(produce_xml_fragment from
https://github.com/django/django/blob/1.4.5/tests/regressiontests/test_utils/tests.py).
The problem seems to not occur with Python 3.2, 3.3 and 3.4.
Before 010b455de0e0:
from StringIO import StringIO
from xml.sax.saxutils import XMLGenerator
stream = StringIO()
xml = XMLGenerator(stream, encoding='utf-8')
xml.startElement(foo, {aaa: 1.0, bbb: 2.0})
xml.characters(Hello)
xml.endElement(foo)
xml.startElement(bar, {ccc: 3.0, ddd: 4.0})
xml.endElement(bar)
stream.getvalue()
'foo aaa=1.0 bbb=2.0Hello/foobar ccc=3.0 ddd=4.0/bar'
After 010b455de0e0:
from StringIO import StringIO
from xml.sax.saxutils import XMLGenerator
stream = StringIO()
xml = XMLGenerator(stream, encoding='utf-8')
xml.startElement(foo, {aaa: 1.0, bbb: 2.0})
xml.characters(Hello)
xml.endElement(foo)
xml.startElement(bar, {ccc: 3.0, ddd: 4.0})
xml.endElement(bar)
stream.getvalue()
''
--
nosy: +Arfrever, benjamin.peterson, larry
priority: normal - release blocker
resolution: fixed -
stage: committed/rejected -
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1470548
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Roundup Robot added the comment: New changeset 010b455de0e0 by Serhiy Storchaka in branch '2.7': Issue #1470548: XMLGenerator now works with UTF-16 and UTF-32 encodings. http://hg.python.org/cpython/rev/010b455de0e0 New changeset 66f92f76b2ce by Serhiy Storchaka in branch '3.2': Issue #1470548: XMLGenerator now works with binary output streams. http://hg.python.org/cpython/rev/66f92f76b2ce New changeset 03b878d636cf by Serhiy Storchaka in branch '3.3': Issue #1470548: XMLGenerator now works with binary output streams. http://hg.python.org/cpython/rev/03b878d636cf New changeset 12d75ca12ae7 by Serhiy Storchaka in branch 'default': Issue #1470548: XMLGenerator now works with binary output streams. http://hg.python.org/cpython/rev/12d75ca12ae7 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka added the comment: Patch updated. Now I get rid of __del__ to prevent hanging on reference cicles as Antoine suggested on IRC. Added test for check that XMLGenerator doesn't close the file passed as argument. -- Added file: http://bugs.python.org/file28797/XMLGenerator-5.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka added the comment: Patch updated. Fixed an error which Georg have found. Restored testing XMLGenerator with StringIO as Antoine pointed. Now XMLGenerator tested for StringIO, BytesIO and an user writer. Added tests for encoding. -- keywords: -easy Added file: http://bugs.python.org/file28724/XMLGenerator-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: patch review - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka added the comment: If no one objects I will commit this next year. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- assignee: - serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Georg Brandl added the comment: I'd like Antoine to have a look at all that io stuff. It looks quite bloated. In your except clause, you're not calling self._close. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka added the comment: If nobody has any objections, why not apply this patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- keywords: +needs review stage: test needed - versions: +Python 3.4 -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka added the comment: Ping. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Eli Bendersky eli...@gmail.com: -- nosy: -eli.bendersky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka storch...@gmail.com added the comment: Here is updated patch with more careful handling of closing (as for issue1767933) and added comments. -- nosy: +eli.bendersky Added file: http://bugs.python.org/file26385/XMLGenerator-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka storch...@gmail.com added the comment: It would be nice to fix this bug before forking of the 3.3.0b1 release clone. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka storch...@gmail.com added the comment: The patch updated to reflect Martin's comments. I hope the old behavior now preserved in the most used in practice cases. Tests converted to work with bytes instead of strings. -- Added file: http://bugs.python.org/file26011/XMLGenerator-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka storch...@gmail.com added the comment: Oh, I see XMLGenerator completely outdated. It even has not been ported to Python 3. See function _write: def _write(self, text): if isinstance(text, str): self._out.write(text) else: self._out.write(text.encode(self._encoding, _error_handling)) In Python 2 there was a choice between bytes and unicode strings. But in Python 3 encoding never happens. XMLGenerator does not distinguish between binary and text streams. Here is a patch that fixes the work of XMLGenerator in Python 3. Unfortunately, it is impossible to avoid the loss of backward compatibility. I tried to keep the code to work for the most common cases, but some code which worked before may break (including I had to correct some tests). -- Added file: http://bugs.python.org/file25760/XMLGenerator.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Serhiy Storchaka storch...@gmail.com added the comment: See also issue1767933. Instead of codecs.StreamWriter better to use io.TextIOWrapper, because the first is slower and has numerous flaws. -- nosy: +storchaka versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Walter Dörwald wal...@livinglogic.de added the comment: An alternative would be to use an incremental encoder instead of a StreamWriter. (Which is what TextIOWrapper does internally). -- nosy: +doerwalter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Mark Lawrence breamore...@yahoo.co.uk added the comment: The are no unit test or doc changes with the patch. Can anyone answer Georg's question on msg66684? -- nosy: +BreamoreBoy versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by Daniel Diniz aja...@gmail.com: -- stage: - test needed type: - behavior versions: +Python 2.6 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1470548 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Georg Brandl [EMAIL PROTECTED] added the comment: Won't this present backwards-compatibility problems if non-ASCII str content is written? -- nosy: +georg.brandl _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1470548 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1470548] Bugfix for #1470540 (XMLGenerator cannot output UTF-16)
Changes by A.M. Kuchling: -- keywords: +easy _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1470548 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
