[sqlalchemy] Re: Again MySQL/unicode - roundtrip failed
On 2006-12-13 22:38, Shannon -jj Behrens wrote: My memory is that MySQLdb recently changed a bunch of stuff and that it was a simple logic bug. You mean, that just the entire logic is reversed? This would explain, why reading is working (from Mike's Mail): def convert_result_value(self, value, dialect): if value is not None and not isinstance(value, unicode): return value.decode(dialect.encoding) else: return value If value is already unicode, then the value is simply handed over (else-part). However, writing goes wrong, because MySQLdb (wrongly) expects an unicode object but gets an utf8 encoded string (if-part): def convert_bind_param(self, value, dialect): if value is not None and isinstance(value, unicode): return value.encode(dialect.encoding) else: return value Am I right? Here's the bug I filed: http://sourceforge.net/tracker/index.php?func=detailaid=1592353grou p_id=22307atid=374932 If I am right, I would add a note to the bug you already filed. Thanks, Stefan -- Start here: www.meretz.de --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Again MySQL/unicode - roundtrip failed
On 12/15/06, Stefan Meretz [EMAIL PROTECTED] wrote: On 2006-12-13 22:38, Shannon -jj Behrens wrote: My memory is that MySQLdb recently changed a bunch of stuff and that it was a simple logic bug. You mean, that just the entire logic is reversed? This would explain, why reading is working (from Mike's Mail): def convert_result_value(self, value, dialect): if value is not None and not isinstance(value, unicode): return value.decode(dialect.encoding) else: return value If value is already unicode, then the value is simply handed over (else-part). However, writing goes wrong, because MySQLdb (wrongly) expects an unicode object but gets an utf8 encoded string (if-part): def convert_bind_param(self, value, dialect): if value is not None and isinstance(value, unicode): return value.encode(dialect.encoding) else: return value Am I right? I can't say with certainty exactly how the code is broken. If you can write a simple, stand-alone test to prove your point, that would indeed be a useful addition to the bug. Remember to use the MySQL client and the hex function to see what's *actually* stored in the database. Here's the bug I filed: http://sourceforge.net/tracker/index.php?func=detailaid=1592353grou p_id=22307atid=374932 If I am right, I would add a note to the bug you already filed. Best Regards, -jj -- http://jjinux.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Again MySQL/unicode - roundtrip failed
in fact its almost definitely a bug in mysqldb - mysqldb should be detecting unicode instances at on the bind parameter side and encoding based on that (otherwise doing nothing), and decoding into unicode instances at the result set level. if it did that, then it would not conflict with the Unicode type on SQLAlchemy's side. here is the source to the SA unicode type: class Unicode(TypeDecorator): impl = String def convert_bind_param(self, value, dialect): if value is not None and isinstance(value, unicode): return value.encode(dialect.encoding) else: return value def convert_result_value(self, value, dialect): if value is not None and not isinstance(value, unicode): return value.decode(dialect.encoding) else: return value as evidence of this, the above Unicode type works completely fine with pysqlite, which also accepts unicode bind params and returns all string values as unicodes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Again MySQL/unicode - roundtrip failed
My memory is that MySQLdb recently changed a bunch of stuff and that it was a simple logic bug. Here's the bug I filed: http://sourceforge.net/tracker/index.php?func=detailaid=1592353group_id=22307atid=374932 On 12/13/06, Michael Bayer [EMAIL PROTECTED] wrote: in fact its almost definitely a bug in mysqldb - mysqldb should be detecting unicode instances at on the bind parameter side and encoding based on that (otherwise doing nothing), and decoding into unicode instances at the result set level. if it did that, then it would not conflict with the Unicode type on SQLAlchemy's side. here is the source to the SA unicode type: class Unicode(TypeDecorator): impl = String def convert_bind_param(self, value, dialect): if value is not None and isinstance(value, unicode): return value.encode(dialect.encoding) else: return value def convert_result_value(self, value, dialect): if value is not None and not isinstance(value, unicode): return value.decode(dialect.encoding) else: return value as evidence of this, the above Unicode type works completely fine with pysqlite, which also accepts unicode bind params and returns all string values as unicodes. -- http://jjinux.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Again MySQL/unicode - roundtrip failed
On 2006-12-12 22:31, Shannon -jj Behrens wrote: When I do it on pure MySQLdb, everything works fine. Not surprisingly, because I explicitly say: cursor.execute(update testtable set test=%s, s) having s as given above (only one row, no where-clause needed). Have you read this thread: http://www.mail-archive.com/sqlalchemy@googlegroups.com/msg00366.html Yes. Sorry, I should have mentioned it. However, I am not sure that this a MySQLdb bug. As I showed using pure MySQLdb the roundtrip works as it should be. When you do on SQLAlchemy: t.test = u's\xfcd!' session.flush() Why is the engine saying: (...) UPDATE testtable SET test=%s WHERE testtable.pkey = %s (...) ['s\xc3\xbcd!', 'first'] Shouldn't it look like this: (...) UPDATE testtable SET test=%s WHERE testtable.pkey = %s (...) [u's\xfcd!', 'first'] Ciao, Stefan -- Start here: www.meretz.de --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
