Tai, The idea behind errors='replace' is that any chars that cannot be properly decoded as unicode will be replaced with an acceptable unicode char (something like '?'). So, if you want to avoid crashing your program on bad input, this is the only acceptable approach and, I assume, why the current approach was taken way back when Django was converted to be all unicode all the time (internally).
However, you bring up an interesting edge case. Apparently the replacement char is causing an invalid SQL statement to be generated - which breaks things in ugly ways. Probably not what we want. I see a couple approaches to this: 1) This could fall under the 'we need better error messages' umbrella and it should be easier to determine what the bad SQL statement was. 2) One could argue that errors='replace' is mangling user input and really an error should be returned to the user, allowing them to fix the error in the form and resubmit. I'm not sure which approach is the way to go here. However, forcing users to deal with encodings is generally a bad idea. Besides, you never can trust a browser to give you what it says it is giving you. In other words, the user may not be able to get the browser to send the correct encoding anyway. For those reasons I'm leaning toward #1. Of course, that begs the question: should Django be doing a better job escaping the data used to build the SQL statement? I guess we won't know unless we get the bad SQL statement. Which takes us back to #1. On Thu, Mar 29, 2012 at 5:42 AM, Tai Lee <real.hu...@mrmachine.net> wrote: > I've just created an essay *cough* I mean ticket about the way Django > silently mangles POST data that is incorrectly encoded, and how this can > send someone (like me) down the rabbit hole trying to debug it, thanks to > "current transaction aborted" database errors. > > This shouldn't normally happen, but it seems that I am unlucky and someone > somewhere is able to get their browser to submit POST data that is encoded > incorrectly. > > I don't want to repeat myself too much and end up writing another essay > here, so I'll just link to the ticket. Just wanted to post this here as well > because jezdez said it might be better suited to discussion on > django-developers. > > https://code.djangoproject.com/ticket/18004 > > Cheers. > Tai. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/django-developers/-/ByLiu7RzHtIJ. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. -- ---- \X/ /-\ `/ |_ /-\ |\| Waylan Limberg -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
