Re: json vs simplejson
On Mon, Jun 11, 2012 at 5:51 PM, Luke Plant wrote: > > i.e. simplejson returns bytestrings if the string is ASCII (it returns > unicode objects otherwise), while json returns unicode objects always. > This seemed strange to me because the standard library json shipping with python 2.7.3 is in fact simplejson 2.0.9, so I did some digging. It turns out that if the C extensions have been compiled and you pass a str instance to loads(), then you get that behavior in both versions. This isn't documented anywhere, but here's the offending pieces: http://hg.python.org/releasing/2.7.3/file/7bb96963d067/Modules/_json.c#l419 https://github.com/simplejson/simplejson/blob/master/simplejson/_speedups.c#L527 If the C extensions aren't enabled, or you pass a unicode string to loads(), then you get the "proper" behavior as documented. I'm not sure how you are triggering this optimized, iffy behavior in django.utils.simplejson though, without also triggering it in the standard library. Did you ever install simplejson with 'pip install simplejson' such that Django picked it up? Can you try running 'from django.utils import simplejson; print simplejson.__version__'? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: json vs simplejson
The other thing this breaks is using **kwargs with something loaded from JSON. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/ynPxXsJZUB8J. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposed Field API additions
On 11/06/12 10:27, Anssi Kääriäinen wrote: > All of the above sounds good - my main worry was that if you subclass > a field, then it will not get a rule match as the module path prefix > will be different than the parent field's. I don't know if this is an > issue even in South... But if the rules are part of the field API, > then there is no worry. Are the rules even needed after the > introduction of "get_init_args()"?. Nope, the whole point of introducing this stuff is to get rid of the rules. That will make me a happy man. > The only possible problem with using the module path as the identifier > is the inability for the user to replace removed fields with copied > backwards compatibility code. As this is probably a non-issue (and > possible to work around) I will surprise you all and not complain > about this :) I think that the case of a field being removed completely is rare enough that this is acceptable (having to use a workaround). Most times a field becomes unimportable it will likely have moved, in which case a good number of apps would keep the old import working (as code probably depends on it). But yes, I'll stop now before you complain again :) Andrew -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Announcement: twice-monthly Django sprints in San Francisco
I've just opened signup for our 2nd sprint - Saturday, June 16th, from 10a to 6p: http://sfdjangosprint2.eventbrite.com/ Please sign up if you plan to attend -- it's free of course. On Tue, May 22, 2012 at 2:30 PM, Jeremy Dunck wrote: > My company, Votizen, now has an office in San Francisco, just a block > from the 4th St Caltrain station. We have room to host sprints - > easily 25 people, though we can grow if there is demand. We have a > professional kitchen, great coffee gear, and maybe a kegerator soon. > There are many great places to eat within an easy walk. > > We will begin hosting day-long, twice-monthly sprints for Django on > the 1st and 3rd Saturdays of each month. We hope that by making it a > comfortable length and a regular occurrence, we can grow a good core > of contribution. > > Note: this first sprint will be on Sunday, June 3rd due to a prior > commitment - sorry about that. :-) > > Since we need to manage space and setup demands in order to make this > a regular event, we'll use eventbrite for free tickets. > > If your company would like to sponsor food & beverage on a per-event > basis, that would be appreciated. Please contact me off-list for > logistics. > > So, if you are in the bay area and would like to contribute, > collaborate, mentor or be mentored, you are invited to attend - please > sign up: > http://sfdjangosprint.eventbrite.com/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
json vs simplejson
Hi all,
We've switched internally from json to simplejson. Our 1.5 release notes
say:
You can safely change any use of django.utils.simplejson to json
I just found a very big difference between json and simplejson
>>> simplejson.loads('{"x":"y"}')
{'x': 'y'}
>>> json.loads('{"x":"y"}')
{u'x': u'y'}
i.e. simplejson returns bytestrings if the string is ASCII (it returns
unicode objects otherwise), while json returns unicode objects always.
This was, unfortunately, a very unfortunate design decision on the part
of simplejson - json is definitely correct here - and a very big change
in semantics. It led to one very difficult to debug error for me already.
So, this is a shout out to other people to watch out for this, and a
call for ideas on what we can do to mitigate the impact of this. It's
likely to crop up in all kinds of horrible places, deep in libraries
that you can't do much about. In my case I was loading config, including
passwords, from a config file in JSON, and the password was now
exploding inside smtplib because it was a unicode object.
Yuck. Ideas?
Luke
--
OSBORN'S LAW
Variables won't, constants aren't.
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization check-in
Hi! This week I managed to write deserialization functions and tests. *Issues with deserialization* Working on deserialization give me a lot thoughts about previous concepts. I rewrite Field class so now Field can't be nested. Field can only have subfields if subfields are attributes: class ContentField(Field): title = Field(attribute=True) # valid content = Field() # invalid -> raise exception in class declaration time def serialized_value(...): ... Of course if ContentField is initialized as attribute and have subfields exception is raised (when ContentField is initialized) I changed python datatype format returned from serializer.serialize method. Previously it was dict with serialized fields (label or field name as key) and special key __attributes__ with dict of attributes. Now it is tuple (native, attributes) where native is dict with serialized fields (or generator of dicts) serializer.deserialize always return object instance After first phase of serialization, python_serialized_object will be serialized by NativeFormat instance. Each format (json, xml, yaml, ...) have one NativeFormat that will translate python_serialized_object to serialized_string. I want to be able to do this: object -> python_serial = object_serializer.serialize(object) -> string_serial = native_format.serialize(python_serial) -> python_deserial = native_format.deserialize(string_serial) -> object2 = object_serializer.deserialize(python_deserial) object2 has same content as object Now I have: object -> python_serial = object_serializer.serialize(object) -> object2 = object_serializer.deserialize(python_deserial) *Tests* I wrote some tests (NativeSerializersTests) for ObjectSerializer in django/tests/modeltests/serializers/tests.py but I'm not sure this is good place for them. I used model (Article) defined in models.py but I used it like normal object. Relation fields aren't serialized in proper way. Until now I tested the most important functions of ObjectSerializer. Creating custom fields, attributes, rename fields (using labels). Next I want to resolve issues with: * Instance creation when deserialize. I have create_instance method and Meta.class_name. I must do some public API from them. * Ensure that Field serialize method returns always simple native python datatypes * Write NativeFormat for (at least) json * Find better names for already defined classes, methods and files * More tests and documentation When I do this serialization and deserialization will be more or less done for (non model) python objects. -- Piotr Grabowski -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposed Field API additions
On 11 kesä, 11:49, Andrew Godwin wrote: > On 10/06/12 21:54, Anssi K ri inen wrote: > > > I agree the needed_fields.py idea was bad. One part of the idea was > > that you would get immediately broken migrations if you remove a > > field. Do all the migrations break if you have a broken field > > reference, or only when you actually use a migration referencing the > > field? > > In the new system all the migrations in the same app would immediately > break (and it would print out a nice explanation showing exactly where > the issue was) - that's a side-effect of deducing the current model > state from the previous migrations, and probably better than only > failing at some mysterious future point. > > > Is there anything else tied to the fields identifier than just how to > > get the field class back when unfreezing? The rules in South seem to > > reference the module path in some way which is why I am asking. > > Well, the module path is in there so you know where to get the field > from - as Luke suggested, that's probably just as good a way of doing it > as registration, and involves not introducing another namespace. > > South's rules also include provision for saying which fields are safe or > unsafe to freeze - that's only because it has to enact this system via > entirely third-party means. Once it's a part of the Field API, checking > if it's there becomes a lot simpler. All of the above sounds good - my main worry was that if you subclass a field, then it will not get a rule match as the module path prefix will be different than the parent field's. I don't know if this is an issue even in South... But if the rules are part of the field API, then there is no worry. Are the rules even needed after the introduction of "get_init_args()"?. The only possible problem with using the module path as the identifier is the inability for the user to replace removed fields with copied backwards compatibility code. As this is probably a non-issue (and possible to work around) I will surprise you all and not complain about this :) - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposed Field API additions
On 10/06/12 21:54, Anssi Kääriäinen wrote: > I agree the needed_fields.py idea was bad. One part of the idea was > that you would get immediately broken migrations if you remove a > field. Do all the migrations break if you have a broken field > reference, or only when you actually use a migration referencing the > field? In the new system all the migrations in the same app would immediately break (and it would print out a nice explanation showing exactly where the issue was) - that's a side-effect of deducing the current model state from the previous migrations, and probably better than only failing at some mysterious future point. > Is there anything else tied to the fields identifier than just how to > get the field class back when unfreezing? The rules in South seem to > reference the module path in some way which is why I am asking. Well, the module path is in there so you know where to get the field from - as Luke suggested, that's probably just as good a way of doing it as registration, and involves not introducing another namespace. South's rules also include provision for saying which fields are safe or unsafe to freeze - that's only because it has to enact this system via entirely third-party means. Once it's a part of the Field API, checking if it's there becomes a lot simpler. Andrew -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
