Hi, I'm having trouble converting an application that uses psycopg2 directly. The Postgres database is encoded in UTF-8. As an example, it has a column called title with the value 'Wilhelm R\xc3\xb6pke', i.e., 'Wilhelm Röpke'. With psycopg2, the connection and retrieval is done more or less as follows:
# -*- coding: utf-8 -*- from psycopg2 import connect from psycopg2.extensions import register_type, UNICODE register_type(UNICODE) db = connect('host=...') curs = db.cursor() curs.execute('select title from tablex') row = curs.fetchone() title = row[0] print(row) print(title) The output from this test program is: (u'Wilhelm R\xf6pke',) Wilhelm Röpke In other words, psycopg2 hands the app a Unicode string, which is ideal because the templating engine (Mako) apparently expects Unicode, even when told the input_encoding is utf-8. If the register_type() is removed, a plain string is delivered but that doesn't help Mako. The comparable SQLAlchemy program is as follows: # -*- coding: utf-8 -*- from sqlalchemy.engine import create_engine engine = create_engine('postgres://...') result = engine.execute('select title from tablex') row = result.fetchone() title = unicode(row.title, encoding='utf-8') print(row) print(title) The output of the SA test program is: ('Wilhelm R\xc3\xb6pke',) Wilhelm Röpke In other words, SA hands the app a plain string instead of Unicode, and if the unicode() conversion is removed, the print(title) (and Mako) complain with: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 9: ordinal not in range(128) I've tried various combinations of convert_unicode, assert_encoding and encoding parameters to the create_engine() call and I have not been able to get Unicode strings from SA, which I find somewhat surprising. Is it possible, or do I have to filter every string retrieved from the database through encoding(value, encoding='utf-8)? For reference, this is on Debian, using Python 2.5.4 and SA 0.5.5. Joe --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---