On Mon, 2008-02-25 at 18:11 -0800, Jean-Pascal Houde wrote: > > > On Feb 25, 5:57 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> > wrote: > > This means that almost certainly you don't, in fact, have correct UTF-8 > > data in your tables. Some ISO-8859-1 codepoints seem to have slipped in. > > You should write a script to run through the data and make sure it > > really is all UTF-8 (and re-encode that stuff that isn't). > > > > Somebody posted here a couple of weeks ago pointing out they had arrived > > in the same situation because their data was created by a PHP app (which > > doesn't seem to do the encoding correctly). > > But all my data was generated by django. At first I was using django > 0.96, which I suppose handle > unicode differently, but I tried exporting the DB, converting it to > utf-8, and reimporting it without success.
So you need to look more at the specifics. When the exception is thrown, look at the exact data that is causing the problem. Work out why it isn't either a unicode object or a string object with UTF-8 encoding (or something like an int that can be safely passed through str()). Looking at the traceback you originally posted, the exception is coming when something is trying to be inserted into the database. Which means somehow some poorly encoded data is getting that far. So try to work out where it is coming from. Add exception handlers to catch the exception and print the details. For example, you could modify the point it leaves Django and enters the MySQLdb backend (the third-last stack frame in your traceback) to print the details of what is being passed in and then trying to trace that back to the source. > Also, I find strange that this work perfectly using the development > webserver, and when I start using fastcgi, > unicode errors come up, what could cause this? Django assumes it is receiving UTF-8 data as the encoding for any form submissions. If that is not true, you need to set the encoding parameter on the request object (see the unicode.txt documentation, in the section called "Form submission"). However, I'd be a little surprised if that was the problem, since we automatically convert to Unicode at that point, so you'd be seeing the problem earlier than you are. Somehow, between the input and when you save to the database, some data that is being passed around as a (byte)string is not UTF-8 encoded. It's also not necessarily latin1 encoded, since Python's latin1 codec is really just a "pass-through" that will map any byte back to itself. > By the way, the "latin1" settings created several other problems for > me, I add to change the default python encoding to > utf-8 to fix some of them, Which is unfortunately code for "I had to break my Python install in subtle ways that will be hard to diagnose." Changing the default encoding is not recommended by any Python developer. Regards, Malcolm -- If it walks out of your refrigerator, LET IT GO!! http://www.pointy-stick.com/blog/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
