Re: [SQLObject] Unicode characters

2008-01-08 Thread Allen Fowler
TED]>; sqlobject-discuss@lists.sourceforge.net Sent: Tuesday, January 8, 2008 8:25:20 AM Subject: Re: [SQLObject] Unicode characters Thanks for all the help! I got it working using the default charset and explicitly setting the necessary columns to UnicodeCol, I am going to try using use_unicode jus

Re: [SQLObject] Unicode characters

2008-01-08 Thread Glenn MacGregor
Thanks for all the help! I got it working using the default charset and explicitly setting the necessary columns to UnicodeCol, I am going to try using use_unicode just to verify. Thanks Glenn On Jan 7, 2008 3:26 PM, Oleg Broytmann <[EMAIL PROTECTED]> wrote: > On Mon, Jan 07, 2008 at 03:13:18P

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 03:13:18PM -0500, Glenn MacGregor wrote: > My mistake, not sure what happened, but when I set the charset=latin1 it > fails in the same place as it fails without the charset set. That is > cursors.py line 146 which encodes the query using latin1 in both cases, > means that m

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
My mistake, not sure what happened, but when I set the charset=latin1 it fails in the same place as it fails without the charset set. That is cursors.py line 146 which encodes the query using latin1 in both cases, means that mysql default charset for the connection is latin1. I am using str everyw

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
BTW, you have said you can insert the string from the command line. But can you do it using MySQLdb? What are correct parameters for connection? Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN.

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 02:38:08PM -0500, Glenn MacGregor wrote: > Changing the connection string to incorporate charset=latin1 fails at a > different place. With that in the string the failure happens at > dbconnection.py line 383. Line 383 in the trunk is return ("INSERT INTO %s (%s)

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
Changing the connection string to incorporate charset=latin1 fails at a different place. With that in the string the failure happens at dbconnection.py line 383. I have also tried charset=utf8 with the same results. I have tried changing the settings in the my.cnf to manage to connection types but

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 02:00:51PM -0500, Glenn MacGregor wrote: > it fails at a different location, line 146 of cursors.py of the Mysqldb > package. > > query = query.encode(charset) > > charset is latin1 and it fails with the error: > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
I really hate to be a pain, but I can not get this to work. I have removed the charset=utf8&use_unicode=1 from the connection string amd get the schema using fromDatabase. When I try to insert the string: Chèrie it fails at a different location, line 146 of cursors.py of the Mysqldb package. que

Re: [SQLObject] Unicode characters

2008-01-07 Thread Dan Pascu
On Monday 07 January 2008, Oleg Broytmann wrote: > On Mon, Jan 07, 2008 at 12:56:04PM -0500, Glenn MacGregor wrote: > > Maybe I need to backup at bit. I am somewhat confused at this point. > > I need to insert a string which contains a non-ascii character into > > my database table, that character

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 12:56:04PM -0500, Glenn MacGregor wrote: > Maybe I need to backup at bit. I am somewhat confused at this point. I need > to insert a string which contains a non-ascii character into my database > table, that character is Ch0xE9rie. E9 is the hex representation of the > acute

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
Maybe I need to backup at bit. I am somewhat confused at this point. I need to insert a string which contains a non-ascii character into my database table, that character is Ch0xE9rie. E9 is the hex representation of the acute-e. Do I need unicode to do this, do I need to change the charset? What a

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 12:35:40PM -0500, Glenn MacGregor wrote: > #!/usr/bin/python > # -*- coding: utf-8 -*- > > import os, sys, glob MySQLdb > from sqlobject import * > > sqlhub.processConnection = connectionForURI('mysql://test:[EMAIL PROTECTED] > /test?charset=utf8&use_unicode=1 > > class T

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
Ok, so I am lost. The debugger is not too much help at this point. I assume this setup should work so we can go from there. -CUT- #!/usr/bin/python # -*- coding: utf-8 -*- import os, sys, glob MySQLdb from sqlobject import * sqlhub.processConnection = connectionForURI('mysql://test:

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 11:43:24AM -0500, Glenn MacGregor wrote: > It looks like somewhere in SQLObject we are calling decode on the variable > which contains Ch?rie to decode it to ascii. Is this the intended result? No, "decoding" in Python means converting from str to unicode. Somewhere (in

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
I do get the schema using fromDatabase. This is a simple table with 2 columns: id, int and label, varchar(64). Everything until now has been ascii, but I now have one string, Chèrie, which I read from a file that I need to add to the db using SQLObject. If I do the insert from the command line int

Re: [SQLObject] Unicode characters

2008-01-07 Thread Oleg Broytmann
On Mon, Jan 07, 2008 at 10:09:51AM -0500, Glenn MacGregor wrote: > ?charset=utf8&use_unicode=1 Do you draw the database scheme from the DB using fromDatabase? > I am reading data from a file, one piece of data contains a char outside of > the ascii range (0xc3). I need to insert that into the

Re: [SQLObject] Unicode characters

2008-01-07 Thread Glenn MacGregor
Oleg, I have been fooling around with this all weekend and I am still unsure of what is going on. In the connections string I now use ?charset=utf8&use_unicode=1, this seems to get past the original error, now I am getting caught elsewhere. When I try to add the offending string to the db I get:

Re: [SQLObject] Unicode characters

2008-01-04 Thread Oleg Broytmann
On Fri, Jan 04, 2008 at 05:08:49PM -0500, Glenn MacGregor wrote: > if isinstance(value, unicode): > -->return value.encode("ascii") I guess you are trying to pass unicode to a StringCol. If this is the case - for unicode use UnicodeCol. Oleg. -- Oleg Broytmann

Re: [SQLObject] Unicode characters

2008-01-04 Thread Glenn MacGregor
Oleg, I am looking in svn at col.py, it seems that from_python in the StringValidator class still uses ascii encoding exclusivly: class StringValidator(validators.Validator): def to_python(self, value, state): if value is None: return None if isinstance(value, uni

Re: [SQLObject] Unicode characters

2008-01-04 Thread Glenn MacGregor
Oleg, Sorry about the top-post. I am using SQLObject 0.10dev-r3137 installed from an egg. Even with the charset set to utf8 I still get the same error. I must be on an older version. I will update and let you know. Thanks On Jan 4, 2008 4:21 PM, Oleg Broytmann <[EMAIL PROTECTED]> wrote: > Ple

Re: [SQLObject] Unicode characters

2008-01-04 Thread Oleg Broytmann
Please, do not top-post. On Fri, Jan 04, 2008 at 04:15:09PM -0500, Glenn MacGregor wrote: > Looking into this issue a bit more revealed that in the StringValidator > class, line 505 on col.py, the from_python function tries to encode all > unicode type strings to ascii. Is this correct behavior? I

Re: [SQLObject] Unicode characters

2008-01-04 Thread Glenn MacGregor
Looking into this issue a bit more revealed that in the StringValidator class, line 505 on col.py, the from_python function tries to encode all unicode type strings to ascii. Is this correct behavior? If so how can I get around it ? Thanks Glenn On Jan 4, 2008 4:08 PM, Glenn MacGregor <[EMAIL P

[SQLObject] Unicode characters

2008-01-04 Thread Glenn MacGregor
Hi All, I am plugging along with Python and SQLObject, very cool stuff. I have a string which contains utf8 (unicode) characters. When I try to add an instance of the table class with that string I get an error: UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 2: ordin