Re: International standard for date and time
On Jun 17, 2007, at 2:46 AM, Jonas wrote: > > On 17 jun, 01:14, "James Bennett" <[EMAIL PROTECTED]> wrote: >> Where do you propose this be used? At the database level, your DB >> will >> store date/time values in whatever internal format it prefers. > Yes, I think that should be stored so at the database level. Databases store dates internally, not strings. Aside from databases being unable to handle dates-times before the epoch this is not a big problem. The only place where ISO formats are appropriate are in machine-readable output (i.e. RSS/ATOM, web services and the like). -- Julian 'Julik' Tarkhanov please send all personal mail to [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 28, 2007, at 7:02 AM, ak wrote: > 1. how do they want to support templates and python code (views/ > scripts) in native encodings if django itself would be all in unicode. `Why do you need to support it, that's the first question that comes to mind. If you have input requirements, decode on read. If you have output requirements, encode on out. > The only way i see is to encode/decode everything at programmer's end > and this means for me no native encodings support at all. AFAIK, templates explicitly should be assumed to be in UTF-8 (supported by most text editors) and decoded as such. Templates being in national encodings should raise nasty and irrecoverable errors all over the place with a proper explanation thereof. > 2. how do they want to support legacy databases if db connection > speaks unicode The database client should support dynamic encoding/decoding (all modern clients do). Not all database backends (AFAIK) properly return unicode strings, so these will have to be scrutinized. People running old clients sack themselves and do the long overdue upgrades. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 27, 2007, at 6:44 PM, ak wrote: > And another thing I still don't understand is: let's pretend I use > MySQL 4.0 with national charset and my templates are in the same > charset too. How would work: It should not work. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 27, 2007, at 6:44 PM, ak wrote: > 1. newforms are with unicode inside > 2. ORM is with str inside 3. welcome to the world of pain -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 26, 2007, at 11:47 AM, Michael Radziej wrote: > # 1356 sets the charset attribute of the mysql backend connection to > 'utf8' for mysql version >= 4.1 And leaves everyone who wants to operate in 8 bits out in the cold. Where they actually ought to be anyway, but I tried to stay liberal in 952 - primarily because it's still unknown how Django authors want to approach this. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 26, 2007, at 11:12 AM, Michael Radziej wrote: > I ask everybody to continue discussion > here in django-developers, and I ask the authors of these three > tickets to work together to find out how to proceed. #952 is the most liberal of all because it does not assume anything about Django's internals, it just tells the binary DB client to decode/encode behind the scenes so that it returns something meaningful (not something the server admin has decided upon two years ago say). -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 26, 2007, at 2:25 PM, Gábor Farkas wrote: > > Julian 'Julik' Tarkhanov wrote: >> >> >> Python's unicode is actually UTF-16 > > on linux it's usually utf-32, and on windows it's usually (always?) > utf-16. sorry I forgot that - it's been a year at least since I last touched Python (actually it was for the Django test drive) > > but you should not care about it. you see, in python, > the unicode-strings are a separate data-type, and there's > just no way to take a bytestring, and tell python: "from now on, > you are an unicode-string, because i know that you are encoded in > utf-16." segregating ustrings and strings is BBD, been' telling it for years. The latest I heard is that the next major Py will abolish bytestrings for good. Getting back to the issue that we were on, I am still strongly advocating the "don't go there" approach for anything but Unicode. How it should be handled in relation to source code is unknown to me (AFAIK Python has a pre-amble sort of declaration that you can actually use to tell the interpreter which encoding your source is in). I just know you hit some major pain when you expect ustrings and get bytestrings instead (and in Python, just as in Perl, only about 30% of the libraries actually care about what they give you). > so while it might be, that the conversion from utf-16-bytestrings to > unicode is sometimes faster thatn converting from utf-8-bytestrings to > unicode, you can't be sure, because as i wrote above, the internal > unicode-encoding is not fixed. > >> whereas IO and the databases mostly >> speak UTF-8 - >> so no, you can't dump it over the wire. > >> We Rubyists are a tad happier >> because we now >> have all in UTF-8 > > you mean that regexes, and all the methods of the string-class now are > unicode-aware in ruby? :) Regexes are unicode-aware for some time already except the case- sensitivity and the class repertoire (which will be fixed when Oniguruma is there). As for the string methods, we mostly took care of them with AS::Multibyte (without silly subclassing) and that works wonders for me. The greatest advantage is that I never have to check what's coming down the pipe because there's only one String to rule them all. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: unicode issues in multiple tickets (#952, #1356, #3370) and thread about Euro sign in django-users
On Jan 26, 1:07 pm, Ivan Sagalaev <[EMAIL PROTECTED]> wrote: > BTW, there were ideas here about really really forcing users to migrate > all data into unicode/utf-8 and be the first guy on the block that would > lead the trend. This is noble but hard and if I remember correctly this > was decided against... Spiteful. Those left behind shall overcome their pain and join. > > > So, for me the quesion sounds like this: either newforms don't use > > unicode to store clean_data and we can keep using 'legacy' character > > sets, or django needs to drop all charsets support except of unicode. > > Or it should convert strings back and forth everywhere LOLIncidentally you > > last 'LOL' is the option that Django have chosen :-). This is about getting expectable bytestrings from the DB, not about unicodifying Django. > 'Unicode' is not a charset, or, more specifically, it is not represented > with bytes. Python's native unicode string represent unicode characters > in some internal format that just can't be dumped over the wire, be it > to database or to the web. Because of this if Django would work > internally in unicode it must encode everything it writes and decode > everything it reads from outside. Converting from unicode to utf-8 is > also encoding, and it does not happen automatically. Python's unicode is actually UTF-16 whereas IO and the databases mostly speak UTF-8 - so no, you can't dump it over the wire. We Rubyists are a tad happier because we now have all in UTF-8 - but if I was working with Django now I would actually _mandate_ the following: - All templates should be UTF-8 (decode on read) - All code should be native Python Unicode (utf16, I don't know how it works with BE-LE but the idea of UTF-16 is really anti-interop) or UTF-8, but I am no Python expert to say whichever is better - All database adapters have to be verified for returning ustrings, and I can ascerain you that most of them won't - Mandate UTF-16 or UTF-8 as client encoding for the database. Does not matter which encoding is used internally because both Postgres and MySQL can now encode/decode on the fly (you will just lose characters if your database is limited) > for charset. Two settings actually: for the web (that we already have) > and for db (that is implemented in #952). I did the #952 when experimenting with Django for my own needs. It's since then abandoned. The solution I made in #952 is the "liberal" one, but I really don't like it - there's need for much more radical solution. Part that solution would be saying to users using old 8-bit crap for code and templates that they are out in the dumps. So feel free to do whatever you find useful with the patch --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: django unicode-conversion, beginning
On 20-aug-2006, at 8:55, Malcolm Tredinnick wrote: >> 5. Internally, work with unicode strings exclusively (after >> transcoding the request and the template). Response should be python >> unicode as well up until the moment it gets sent out. > > That's the idea. Not so fast. You want to be liberal and send out BIG5 and JIS output, but at the same time use Unicode strings on the inside. How are you going to represent the characters which you want to preserve and handle specially with these Asian encodings if all you have in the machinery is Unicode? If you can't handle these characters then what is the point of having switchable output and input? Are there browsers that don't handle UTF-8? I mean, modern ones. Even Lynx does it properly. How are you going to encodiUriCompnents in JS with other charsets? Encode URIs? > Metaphorically cutting off both our arms so that we appear > more aerodynamic is probably not a gain worth making. I don't agree, but I rest my case. I just thought UTF-8 is the optimum compromise and enough non-conformity already. Thought that Django can be one of those frameworks that cut the knot instead of spending weeks unwinding it. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: django unicode-conversion, beginning
On 17-aug-2006, at 1:08, Bill de hÓra wrote: > like wanting to serve utf8 rss feeds, but have latin1 come > in and out of mysql. Might seem very extreme, but I would love to chime in. Maybe it would be wise to go even further, whereby: 1. Hardcode Django to output and input UTF-8 as the most useful for interop 1a. Any case where the developer might expect different input (for instance almost all OPML files are still exported as ISO due to idyosyncrastic way Radio worked back in the day) has to be known to him and handled explicittly 1b. Honor the charset headers sent in the request for transcoding 1c. Allow everyone who wants to output other charsets to cry and perish. 2. Stick the utf-8 output charset anywhere where it's possible (headers, page head...). 2. Allow the DB to be in another encoding for databases that support it. For instance, MySQL and Postgress will transcode the strings for the client on the fly, so you can do interop with them in UTF-8 even when they are in a different encoding. 3. Assume all templates are in UTF-8 as well because text editors have much more success dealing with it them that way. Transcode templates on read into unicode strings. 4. As a consequence of 1, let DEFAULT_CHARSET go. Too many choices really hurt here. 5. As a consequence of 1, deprecate the DATABASE_CHARSET I sent in as a patch and make it the default, so that all drivers switch their database clients to the most suitable Unicode form. SQLite has to be compiled with Unicode support, this has to be mentioned in the docs. 5. Internally, work with unicode strings exclusively (after transcoding the request and the template). Response should be python unicode as well up until the moment it gets sent out. Important to note is that every database driver has to be scrutinized for whether it returns unicode strings proper. I know, it seems so nice to be liberal and allow people to choose their encoding but just too many situations prove that to be the Wrong Choice. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: urlify.js blocks out non-English chars - 2nd try?
On 7-jul-2006, at 17:50, Bill de hÓra wrote: > This is my point. Cut what exactly? "No good" for what exactly? We > could file patches to see what sticks, but it might be better to > figure > what's wanted first, instead of playing fetch me a rock. This is handled by Unicode standard and is called transliteration. The problem is that it's locale dependent. AFAIK Python's codecs don't implement it (but ICU4R does). If you go for tables it's going to be _many_. URLs can be Unicode-aware, just encoded - so why not replacing whitespace with dashes and doing a Unicode downcase, and be done with it? Some browsers (Safari) even show you the request string verbatim, so it's very readable. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: the so-called [AUDIT]
On 2-jun-2006, at 22:14, Carlo C8E Miron wrote: > > On 6/2/06, Julian 'Julik' Tarkhanov <[EMAIL PROTECTED]> > wrote: >> >> I would advise all respected Django contributors to follow the path >> mentioned here: >> >> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/138966 > > and maybe also > > http://www.encyclopediadramatica.com/index.php/Ilias Quite a history: http://dev.eclipse.org/newslists/news.eclipse.foundation/msg00291.html http://www.intellij.net/forums/thread.jspa? forumID=27&messageID=3439721&threadID=122131&q= and a response from Hani is the best to date, hats off. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: the so-called [AUDIT]
On 2-jun-2006, at 22:14, Carlo C8E Miron wrote: > > On 6/2/06, Julian 'Julik' Tarkhanov <[EMAIL PROTECTED]> > wrote: >> >> I would advise all respected Django contributors to follow the path >> mentioned here: >> >> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/138966 > > and maybe also > > http://www.encyclopediadramatica.com/index.php/Ilias Half of Google to be precise. Subject closed. -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: the so-called [AUDIT]
I would advise all respected Django contributors to follow the path mentioned here: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/138966 On 25-mei-2006, at 18:27, lazaridis_com wrote: > > For a long term project, I am selecting a python based persistency > system: -- Julian 'Julik' Tarkhanov please send all personal mail to me at julik.nl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: MySQL 4.1.x charset support
http://code.djangoproject.com/ticket/952 On 22-mrt-2006, at 6:54, ak wrote: > > Thanks, posted: http://code.djangoproject.com/ticket/1528 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: MySQL 4.1.x charset support
I file a much broader patch for thus about a few months ago. Check out the tickets. On 22-mrt-2006, at 6:54, ak wrote: > > Thanks, posted: http://code.djangoproject.com/ticket/1528 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
