Hi,
I came across thoes tickets and the corresponding thread just yesterday and as
fas as I understood the main problem is that newforms ist talking unicode
internally and at the interface to the django-ORM.
I attached my solution to this problem for django.newforms.models (diffed
against
Adrian Holovaty wrote:
> Hi Ivan,
>
> Could you explain again why you think newforms should output
> clean_data as bytestrings rather than Unicode strings?
I don't think so :-). Quite the opposite. I think it's good that you
made newforms in unicode since it effectively gives a start to
Adrian Holovaty:
> On 2/15/07, Ivan Sagalaev <[EMAIL PROTECTED]> wrote:
>> I tried to show that it leaves out only two things:
>>
>> - if DEFAULT_CHARSET is different than DB charset it won't work (but
>> it's a weird situation, most legacy systems have one legacy encoding for
>> both)
>>
>> - it
On 2/15/07, Ivan Sagalaev <[EMAIL PROTECTED]> wrote:
> I tried to show that it leaves out only two things:
>
> - if DEFAULT_CHARSET is different than DB charset it won't work (but
> it's a weird situation, most legacy systems have one legacy encoding for
> both)
>
> - it doesn't help if unicode
Michael Radziej wrote:
> (meeting postponed ...)
Nice :-)
> You're right, sorry. I was in a different ticket and somehow thought
> it was the same.
>
> Yes, #3370 looks interesting and is a different solution. I'm not
> sure whether it deals with all the issues of this thread.
I tried to show
Ivan Sagalaev:
> Michael Radziej wrote:
>> Ivan Sagalaev:
>>
>>> Michael, the ticket http://code.djangoproject.com/ticket/3370 just got a
>>> patch that does a) and it's really small. It's not as full as having b)
>>> and d) but I think they are really a corner cases: b) for different
>>>
Michael Radziej wrote:
> Ivan Sagalaev:
>
>> Michael, the ticket http://code.djangoproject.com/ticket/3370 just got a
>> patch that does a) and it's really small. It's not as full as having b)
>> and d) but I think they are really a corner cases: b) for different
>> encodings in DB and in
Michael Radziej wrote:
> A few days ago, I wrote:
>> I see three ways to fix the problem in #3370:
>>
>> a) newforms stops passing unicode strings to the Database API and uses
>> bytestrings.
>>
>> b) the database wrapper in Django sets connection.charset (but needs to
>> translate the charset
On Feb 1, 4:16 pm, Michael Radziej <[EMAIL PROTECTED]> wrote:
> Ivan Sagalaev:
>
> > Michael Radziej wrote:
> >> d) make the database wrapper accept both unicode and bytestrings in
> >> the models, but always pass unicode strings to the database backend.
Sounds like a reasonable proposal. You
On Jan 27, 2007, at 6:44 PM, ak wrote:
> And another thing I still don't understand is: let's pretend I use
> MySQL 4.0 with national charset and my templates are in the same
> charset too. How would work:
It should not work.
--
Julian 'Julik' Tarkhanov
please send all personal mail to
me at
On Jan 27, 2007, at 6:44 PM, ak wrote:
> 1. newforms are with unicode inside
> 2. ORM is with str inside
3. welcome to the world of pain
--
Julian 'Julik' Tarkhanov
please send all personal mail to
me at julik.nl
--~--~-~--~~~---~--~~
You received this
Hi,
A few days ago, I wrote:
> I see three ways to fix the problem in #3370:
>
> a) newforms stops passing unicode strings to the Database API and uses
> bytestrings.
>
> b) the database wrapper in Django sets connection.charset (but needs to
> translate the charset name since the databases
Bill de hOra wrote:
> Yep; it's a problem on the way back in. Python won't let you interpolate
> encoded bytestrings and unicode; you have to state the encoding. Ivan -
> could the db encoding be declared in settings.py?
This is what #952 is about. Though it doesn't convert things for DB on
Ivan Sagalaev wrote:
> Michael Radziej wrote:
>>
>> I don't see a problem with the generic views since they pass bytestrings
>> to the database wrapper, this gets as bytestrings to MySQLdb, and for
>> bytestrings the charset attribute is not used at all.
>
> Umm... This is the exact problem with
Michael Radziej wrote:
> I thank you for all your patience with me. I was completely off-track. I
> read all the mails again, and everything is starting to make sense now.
Then I hope not to confuse you (and everyone else) with my answer :-)
> First, contrary to my former opinion, #3370 is a
Hi there,
I thank you for all your patience with me. I was completely off-track. I
read all the mails again, and everything is starting to make sense now.
This is going to be a lengthy email about #1356 and #3370, but please do
read until the end. Short executive summary: It's really a bug, and
[EMAIL PROTECTED] wrote:
> I think the next step in the unicodeification of django is to decide where
> the conversions happen. Or has this already been decided?
>
> I like the picture of "unicode circle of trust": everything inside the circle
> is trusted as unicode strings. Everything
Michael Radziej wrote:
> Hey, I now finally understand why you need #952 as soon as you switch to
> a different charset. I understand your point, but I'd rather offer a
> solution than postponing this for such a long time.
+1
#952 is good to include now since it plays nice with byte string
Hi,
Ivan Sagalaev schrieb:
> Michael Radziej wrote:
>> I'm not sure about what the last sentence means--are you suggesting to
>> put #3370 (the mysql part) into "Needs design decision"?
>
> ## 3370
>
> I'm -1 on setting MySQL connection to 'utf8' in #3370. It *will* make
> sense when we will
Ok, thanks for that Ivan,
Michael - ignore what I said before :-).
The real question, then, is what will it take to get Django unicode
uh, "safe" (not sure if that's the best term) before 1.0. I realise
that this looks like it's going to be fairly major to sort out, but if
we don't then
Michael Radziej wrote:
> I'm not sure about what the last sentence means--are you suggesting to
> put #3370 (the mysql part) into "Needs design decision"?
## 3370
I'm -1 on setting MySQL connection to 'utf8' in #3370. It *will* make
sense when we will have newforms ready and models containing
ak wrote:
> Bjorn, if you read my first messages and specially my patch #3370, you
> find that I made a suggestion that if the guys want to move to unicode
> they better drop all native encodings support and so does my patch.
With all due respect, you seem to not understand this. 'Unicode'
Hi Simon,
[EMAIL PROTECTED]:
> +1
>
> I was just coming to the same conclusion - #952 looks good to go, and
> #3370 could be split into the __repr__ and mysql issues. __repr__ and
> #952 are easy to solve. The rest of it needs the cores to come to a
> decision about this.
I'm not sure about
+1
I was just coming to the same conclusion - #952 looks good to go, and
#3370 could be split into the __repr__ and mysql issues. __repr__ and
#952 are easy to solve. The rest of it needs the cores to come to a
decision about this.
--Simon
Hi,
ak schrieb:
> After some thoughts I came to the following conclusion: if you guys
> want to keep support of legacy charsets in fact you don't have to
> force model objects too be unicoded. Firstly, they are passed to
> templates and filters and we can't mix legacy charsets with unicode in
On Jan 28, 2:02 pm, "ak" <[EMAIL PROTECTED]> wrote:
> Bjorn, if you read my first messages and specially my patch #3370, you
> find that I made a suggestion that if the guys want to move to unicode
> they better drop all native encodings support and so does my patch.
You mean require all I/O
After some thoughts I came to the following conclusion: if you guys
want to keep support of legacy charsets in fact you don't have to
force model objects too be unicoded. Firstly, they are passed to
templates and filters and we can't mix legacy charsets with unicode in
one template. Next, if
Michael Radziej wrote:
> 1. Are all these tickets really about the connection encoding?
>
> 2. If so, what's the problem of using utf8 for the connection for
> everybody? I don't see how this would be a problem for anybody who is
> using a different encoding for templates, within the database's
ak wrote:
> Could someone please explain me what was a problem with unicode support
> in oldforms so newforms have been made with unicode inside ?
I can! The thing is it has absolutely nothing to do with forms, it's
just historical coincidence.
Originally Django was written with using byte
Michael, of you read again the topic about euro sign in newforms you
can find that this touches everything. Personally I couldn't find a way
to use utf-8 to connect MySQL and keep using cp1251 in my templates: it
basically doesn't work. With my patch (#3370) and utf8 everywhere it
does.
Guys
Could someone please explain me what was a problem with unicode support
in oldforms so newforms have been made with unicode inside ?
Kick me if I wrong but what is a real reason to convert bytes back and
forth ? Religion ? I agree with everyone who says that unicode is a
must and 'legacy'
On Jan 26, 2007, at 11:47 AM, Michael Radziej wrote:
> # 1356 sets the charset attribute of the mysql backend connection to
> 'utf8' for mysql version >= 4.1
And leaves everyone who wants to operate in 8 bits out in the cold.
Where they actually ought to be anyway, but I tried to stay
On Jan 26, 2007, at 2:25 PM, Gábor Farkas wrote:
>
> Julian 'Julik' Tarkhanov wrote:
>>
>>
>> Python's unicode is actually UTF-16
>
> on linux it's usually utf-32, and on windows it's usually (always?)
> utf-16.
sorry I forgot that - it's been a year at least since I last touched
Python
Julian 'Julik' Tarkhanov wrote:
>
>
> Python's unicode is actually UTF-16
sorry, but no. it's not utf-16.
it's decided at compile-time,
and i'ts either utf-32 or utf-16.
on linux it's usually utf-32, and on windows it's usually (always?) utf-16.
but you should not care about it. you see,
ak wrote:
> Ticket http://code.djangoproject.com/ticket/952 contain a complete
> solution of this problem and I don't know why it was not merged into
> the code but at the moment it is not matter and here is the reason why:
> Since newforms library was born and the decision about using unicode
>
Hi,
here's a summary what the different tickets are about:
# 952 adds a database client encoding setting,
DATABASE_CLIENT_CHARSET, for mysql and postgresql backends. For
mysql, it uses the given charset in 'SET NAMES' to build the
connection, except for mysql < 4.1. For postgresql, it does a
Guys
The problem is simple but it was born a very long time ago.
For MySQL 4.1 and higher there is hardcoded in
django/db/backends/mysql/base.py:
cursor.execute("SET NAMES 'utf8'")
there were lots of tickets and messages in django-users complaining to
this but in fact they all were ignored.
Michael Radziej wrote:
> Hi,
>
> we have a bit of chaos here ... Tickets 3370, 1356 and probably 952
> all are about this problem, all are accepted, and #3370 and #1356
> have very similar patches. I ask everybody to continue discussion
> here in django-developers, and I ask the authors of these
38 matches
Mail list logo
