Hi All
I thought I had "Character Encoding" licked, but I've hit something I can't
work through. Any help appreciated.
I have a "legacy" non SQL database that I read legacy data from (using cool
Python code that emulates the old ISDB binary comms) and it reads a str which
has "Foreign" language chars in it. (French for example).
So, firstly, I have myStr = ''Aligot\xc3\xa9" which when printed is Aligoté.
So far so good.
I then convert that to unicode by myUnicode = unicode(myStr, 'utf-8',
errors='ignore') and get u'Aligot\xe9'. This printed is also Aligoté,
therefore all is good.
I have a MySQL database, InnoDB table, charset utf-8.
I set up my values in a dict called setValues with all the columns and their
respective unicode'd values ready to go
I then do a table.insert(values=setValues).execute() and get this error.
Traceback (most recent call last):
File "C:\Documents and Settings\wprince\Desktop\PY CODE
DEVELOPMENT\CESyncSQL\TEST_Sync.py", line 148, in SYNC_IT
SyncFunction(ceDB, session, meta)
File "C:\Documents and Settings\wprince\Desktop\PY CODE
DEVELOPMENT\CESyncSQL\TEST_Sync.py", line 840, in SYNC_VarietiesOUT
DAPDB_SetColumns(meta, 'varieties',
{'DescriptiveText':self.CEUnicode(tVarieties.ceVarietyText.value),
'FlavourText':self.CEUnicode(tVarieties.ceFlavourText.value),
'ImageURL':imageURL}, Variety=variety)
File "C:\Python26\lib\DAPDBHelpers.py", line 323, in DAPDB_SetColumns
table.insert(values=setColumns).execute()
File "C:\Python26\lib\site-packages\sqlalchemy\sql\expression.py", line 1217,
in execute
return e._execute_clauseelement(self, multiparams, params)
File "C:\Python26\lib\site-packages\sqlalchemy\engine\base.py", line 1722, in
_execute_clauseelement
return connection._execute_clauseelement(elem, multiparams, params)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 4:
ordinal not in range(128)
I know what the error "means", I just don't know why I'm getting it. The
offending u'\xe9' character is in the DescriptiveText column. DAPDB_SetColumns
is a simple wrapper around an update/insert that builds up the
table.insert(values=setColumns).execute() you see.
This is what setColumns looks like;
{'ImageURL': '', 'DescriptiveText': u'Carm\xe9n\xe8re is a red wine grape
variety originally from Bordeaux, France. Having lost favor in France, the
largest area planted with this variety is in now Chile. It only survived, due
to growers believing it was Merlot. The vines were imported into Chil',
'FlavourText': u'Carmenere is a full bodied red wine with approachable tannins
and a combination of sweet berry fruit, savory pepper, smoke, tar, with a
slight leafy character.\n', 'Variety': u'Carmenere'}
'Variety' is the primary key BTW.
What gives? It feels like SQLA is encoding/decoding somewhere it shouldn't..
Cheers
Warwick
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
