Re: Denormalisation Magic, Round Two

[Ben Welsh]

Just want to step out of lurker mode and pipe up with my modest +1.

I've been able to hack together simple denorm tricks for FK fields and
static data that work something like these fields -- but one thing I could
really use some help with yet is pulling off denormalization of frequently
changing M2M fields. Do y'all think that's something that will have to wait
for M2m signals like ticket 5390
or can modifications to the
methods here make it work in 1.0? I haven't been
able to square the circle in my feeble brain, so here I am.

Love what you're doing,

Ben.



On Sun, Nov 30, 2008 at 3:33 PM, Christian Schilling <
initcr...@googlemail.com> wrote:

>
> On Nov 30, 9:55 pm, Andrew Godwin  wrote:
> > I have a Trac setup around on my server for South, so if you want I'll
> > kick that slightly and install the Git plugin...
>
> this should be  easier than setting it up on my server where i run
> debian/stable which contains only an old version of trac and no trac-
> git at all.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Christian Schilling]

On Nov 30, 9:55 pm, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> I have a Trac setup around on my server for South, so if you want I'll
> kick that slightly and install the Git plugin...

this should be  easier than setting it up on my server where i run
debian/stable which contains only an old version of trac and no trac-
git at all.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

Christian Schilling wrote:
> yes, i merged that, ran my old tests against the fixed version and
> found a bug in the fix..
> so, i recreated the scenario in the new testsuite and fixed the
> fix ;-)
>   

Ah, the wonderful meta-loop of fix fixes.

> i think we should really setup a trac instance somewhere...

I have a Trac setup around on my server for South, so if you want I'll
kick that slightly and install the Git plugin...

Andrew



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Christian Schilling]

On Nov 30, 12:54 am, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> The unit test 'suite' is hardly finished yet (one test...), but it does
> at least do a pretty thorough use check of some of the more basic
> scenarios. ./manage.py test denorm does indeed test and come out OK,
> though - it even brought up an infinite recursion bug I also had in my
> code, which I've fixed (there may be a cleaner fix).
>

yes, i merged that, ran my old tests against the fixed version and
found a bug in the fix..
so, i recreated the scenario in the new testsuite and fixed the
fix ;-)

i think we should really setup a trac instance somewhere...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

Christian Schilling wrote:
> I think the problem here is that you need to alter a model after
> django has initialized it, because there is just no other way to auto-
> detect
> the field-type of the mirrored field. So eliminating the need to type
> both the related model name as well as the field name is not hard at
> all,
> syntax could look like:
>
> owner_username = denormalized(models.CharField,max_length=100)(Mirror
> ('owner','username'))
>
> so Mirror instances would be a callable with the proper dependency
> attached to it.
> Eliminating the need for the first line, where you specify the field
> type
> might be impossible to do in a clean way.
>   

Yeah, it pretty much is. I did it in my original code, but only by
hooking into class_prepared to then traverse across the foreign key, and
_then_ dynamically reparent the field class to the right parent. It's
python, so it's possible, but that doesn't mean it's sensible. Besides,
I'm not sure all Python implementations like you fiddling with __bases__.

> This is relevant for me to, as i use south for my migrations.
>   

Ooh, excellent :) I've fixed this, anyway; South has provision for more
magical fields, and if they have a south_field_definition function will
use that instead of source file scanning, so one more method on the
DBField subclass and it's playing very nicely. I am now happy.

> This would be great, my current test coverage is rather small and
> better/more
> tests will cerainly be exremly usefull when trying to optimise the
> code.
>   

The start of this and the South stuff is over on my branch:

http://github.com/andrewgodwin/django-denorm/tree/master

The unit test 'suite' is hardly finished yet (one test...), but it does
at least do a pretty thorough use check of some of the more basic
scenarios. ./manage.py test denorm does indeed test and come out OK,
though - it even brought up an infinite recursion bug I also had in my
code, which I've fixed (there may be a cleaner fix).

Andrew



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Christian Schilling]

On Nov 28, 5:30 pm, Andrew Godwin <[EMAIL PROTECTED]> wrote:
>
> One: This is the MirrorField equivalent in the example:
>
>     @denormalized(models.CharField,max_length=100)
>     @depend_on_related(User)
>     def owner_username(self):
>         return self.owner.username
>
> While it's really obvious what it's doing, I (for obvious reasons) would
> like to have common use cases like that shortened, and the number of
> things I have to type reduced (I only need one of the other model or the
> foreign key attribute name, and here I need both). I've tried writing a
> wrapper for MirrorField, but I'm running into the same issue I had
> (which is that you have to do it on the class_prepared signal). I'll
> probably have it working soon, though.
I think the problem here is that you need to alter a model after
django has initialized it, because there is just no other way to auto-
detect
the field-type of the mirrored field. So eliminating the need to type
both the related model name as well as the field name is not hard at
all,
syntax could look like:

owner_username = denormalized(models.CharField,max_length=100)(Mirror
('owner','username'))

so Mirror instances would be a callable with the proper dependency
attached to it.
Eliminating the need for the first line, where you specify the field
type
might be impossible to do in a clean way.

> The second thing is that it doesn't play nice with South, but that's
> almost certainly my fault, and nothing to do with you. If we hadn't
> started scanning source code for the field definitions and ripping them
> out it might have been fine, but it's proved to be the only real way...
This is relevant for me to, as i use south for my migrations.

> Still, 'tis nice overall. I have a partially-complete unit test suite
> for my denorm code I'd be happy to port over (it has the wonderful set
> of tricks you need to unit test with a whole fake app, while still
> keeping database settings) if you want.

This would be great, my current test coverage is rather small and
better/more
tests will cerainly be exremly usefull when trying to optimise the
code.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

So, I've had a good look over the (constantly-growing!) code here, and
I'm really beginning to like the decorator approach, especially the nice
way you combine dependencies with the dependency classes, which makes my
solution to old/new FK values look terrible.

I still have two issues, although I'm pretty sure both can be fixed.

One: This is the MirrorField equivalent in the example:

@denormalized(models.CharField,max_length=100)
@depend_on_related(User)
def owner_username(self):
return self.owner.username

While it's really obvious what it's doing, I (for obvious reasons) would
like to have common use cases like that shortened, and the number of
things I have to type reduced (I only need one of the other model or the
foreign key attribute name, and here I need both). I've tried writing a
wrapper for MirrorField, but I'm running into the same issue I had
(which is that you have to do it on the class_prepared signal). I'll
probably have it working soon, though.

The second thing is that it doesn't play nice with South, but that's
almost certainly my fault, and nothing to do with you. If we hadn't
started scanning source code for the field definitions and ripping them
out it might have been fine, but it's proved to be the only real way...

Basically, I'm moaning as it doesn't work with my peculiar setup :)
Still, 'tis nice overall. I have a partially-complete unit test suite
for my denorm code I'd be happy to port over (it has the wonderful set
of tricks you need to unit test with a whole fake app, while still
keeping database settings) if you want.

Andrew



Christian Schilling wrote:
> i spend some time implementing my idea above (still just a proof of
> concept, all testing i did was on the example project)
> the resulting models.py: (still in the same place)
> http://github.com/initcrash/django-denorm/tree/master/example%2Fgallery%2Fmodels.py
>
> as you can see, this makes the @denormalized decorator a drop-in
> replacement for pythons @property
> decorator. and the function becomes much shorter and easier to
> understand.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Christian Schilling]

i spend some time implementing my idea above (still just a proof of
concept, all testing i did was on the example project)
the resulting models.py: (still in the same place)
http://github.com/initcrash/django-denorm/tree/master/example%2Fgallery%2Fmodels.py

as you can see, this makes the @denormalized decorator a drop-in
replacement for pythons @property
decorator. and the function becomes much shorter and easier to
understand.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

Christian Schilling wrote:
> i don't think so.
> if you mean lines 35-38 in fields.py:
> this is only used to rebuild all denormalized values in the whole DB
> via the management command, witch means everything needs to be updated.
>   

Ah yes, ignore me. I was trying to see how you did 'efficient updates'
(where, say, only half of a model's instances are affected by the change
that just occurred) but I now see it's the function's job to do that.

I quite like this solution, it's rather clean; my own solution is far
more lengthly, and while some of that is the efficiency code I
previously mentioned, I can't help but think I have some redundancy in
there now...

I've published my code at http://code.google.com/p/django-denorm/, and
I'll 'release' it soonish, but I'm liking this decorator approach more
and more, especially as we can then just build 'AggregateFields' on top
of them.

Andrew

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Christian Schilling]

one more thing...

On Nov 23, 12:00 pm, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> Additionally, your code loops over every model every time, whereas in
> some cases you can detect which specific model needs updating (for
> example, if you know that the field "planet" on the just-saved object is
> a foreign key to your model).

i don't think so.
if you mean lines 35-38 in fields.py:
this is only used to rebuild all denormalized values in the whole DB
via the management command, witch means everything needs to be updated.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Christian Schilling]

I think it would be even nicer if the function passed into the
decorator could
actualy return the new value of the field, instead of assigning it and
call save().
Currenty the function does two tasks:
1) figure out what needs to be updated "resolve dependenys"
2) calculate the new value
maybe the first task could be implemented in such a generic way, that
it fits
most cases.
Maybe something like a DependencyResolver class that has a method
witch takes
an instance (the one beeing saved) and returns a queryset (instances
that need to be
updated.
this could result in a syntax like this:

@denormalized(models.TextField,blank=True,depend=
['self',DirectForeignKey(Picture)])
def users():
return ', '.join(str(p.owner) for p in self.picture_set.all())

so DirectForeignKey (has DependencyResolver as base class) could look
for a FK to Gallery in Picture.
this way the most common cases would require very litte code, and if
somebody has
some very uncommon dependencies he can implement a special resolver.
i'll some experimenting on implementing this.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

Hmm, now that's an interesting implementation. I'm pleased to see you've
essentially taken the same route I did in terms of detecting when to
save, which means I'm not being as silly as I first thought.

The decorator syntax is nice, and certainly something that should be
around - however, I (naturally) feel that the most common cases - i.e.
copying values across, and count() queries - should have an actual field
people can use, to lower the barrier to entry.

Additionally, your code loops over every model every time, whereas in
some cases you can detect which specific model needs updating (for
example, if you know that the field "planet" on the just-saved object is
a foreign key to your model).

Still, I certainly like the idea of having a decorator syntax; I'm going
to go away and see if I can persuade my AggregateField to turn
inside-out like that.

Andrew


[EMAIL PROTECTED] wrote:
> i just implemented something like this. it useses decorator-syntax
> to create a denormalized field from a callable.
> i think the advantages of this aproach are:
> 1) it's very flexible
> 2) all code related to the denormalisiation is located in one central
> place,
> and not one place for the field definition and another place for the
> actual code
>
> i also implemented a management command for rebuilding all the
> denormalisations.
> a (working) example:
> http://github.com/initcrash/django-denorm/tree/master/example/gallery/models.py
>
> the actual implementation of the decorator contains quite a few ugly
> hacks, maybe
> this can be done in a cleaner way by changing some code inside django
> itself.
> http://github.com/initcrash/django-denorm
> >   


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[[EMAIL PROTECTED]]

i just implemented something like this. it useses decorator-syntax
to create a denormalized field from a callable.
i think the advantages of this aproach are:
1) it's very flexible
2) all code related to the denormalisiation is located in one central
place,
and not one place for the field definition and another place for the
actual code

i also implemented a management command for rebuilding all the
denormalisations.
a (working) example:
http://github.com/initcrash/django-denorm/tree/master/example/gallery/models.py

the actual implementation of the decorator contains quite a few ugly
hacks, maybe
this can be done in a cleaner way by changing some code inside django
itself.
http://github.com/initcrash/django-denorm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

David Cramer wrote:
> I'm not sure on AggreateField either. What if you just do like
> ("Photo", "user__exact=self.user") or something. Currently there's no
> rerepsentation for "self" in Django QuerySet's, so this is a hard
> thing to call. Also, we need a way to support callables.
>
> e.g. models.MirrorField("my_callable_or_attribute") as well as the
> models.MirrorFIeld("my_attribute", "its_subattribute")
>   

Well, callables are a lot harder to deal with; there's no efficient way
to hook onto when they've changed. It would be possible if you passed
along a foreign key to watch for changes, although at that point you
could probably also implement it using an AggregateField of some kind,
since they already take arbitary aggregate functions.

The problem is the 'self' thing, though. However, I like the
filter()-style syntax, I'll need to see if that's easy enough to roll in
so it still makes sense...

Andrew


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[David Cramer]

I'm not sure on AggreateField either. What if you just do like
("Photo", "user__exact=self.user") or something. Currently there's no
rerepsentation for "self" in Django QuerySet's, so this is a hard
thing to call. Also, we need a way to support callables.

e.g. models.MirrorField("my_callable_or_attribute") as well as the
models.MirrorFIeld("my_attribute", "its_subattribute")

On Oct 27, 6:26 am, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> Russell Keith-Magee wrote:
> > I do have a slight reservation regarding the API you are proposing.
> > The fields you have proposed (MirrorField and AggregateField) capture
> > the obvious use cases, but the syntax don't feel consistent with the
> > rest of the Django API. In particular, AggregateField seems to be
> > introducing a whole new query syntax that is completely independent of
> > the aggregation syntax that is due for inclusion in v1.1.
>
> > I would prefer to see a syntax that demonstrated slightly better
> > integration with the Django query syntax. If it isn't possible to use
> > QuerySet itself as a way to develop the underlying query, then I would
> > expect to see a syntax that at the very least retained the flavour of
> > the Django query syntax rather than inventing "fk:" style operators
> > that have no real analog.
>
> Indeed; I was far happier with the queryset syntax for AggregateField I
> originally proposed, but I couldn't get it to work - it just won't go
> inside class bodies without far, far too much hackery. The MirrorField
> API I like more; I have an alternate version where you simply pass it
> the model name and column name, but that then has issues if you have
> more than one FK to the same model.
>
> The AggregateField proposal is just that, and I really squirmed at the
> idea of "all:" and "fk:", but it got it working pretty quickly. One
> possibility for more clean integration is an approach more like
> querysets, so you could have something like
>
> photos_count = models.AggregateField(Photo.view.filter(user_self=True))
>
> The main problem with anything like this (and indeed with my attempt to
> implement this with plain old querysets) is that, as a field, you exist
> only on a class, and so when you get a signal saying a model has been
> updated, it's somewhat difficult to determine which instances of
> yourself to update - you can't loop through them all testing, since
> that's hilariously inefficient. That's why I limited the querying to
> only fk: and all: types, since detecting these is far more efficient.
>
> I'd love to see any proposal for how Aggregates should look, as my
> current incarnation really isn't too django-ish. I personally like
> MirrorFields (now on their third name, more welcomed) as I have them
> now, but then I would, since I invented them for my own use...
>
> Andrew
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

A Mele wrote:
> I have followed the discussions about denormalization and I really
> would like to see this in trunk. I agree with Rusell, the syntax
> should be more like Django's query syntax, but I think this kind of
> field is very useful for a lot of projects.
>   

I obviously agree, but Russ' point about it being ugly still stands, and
the only improvement I've been able to think of so far is changing
"fk:foo" into fk="foo", so you get something like:

models.AggregateField("Person", fk="nation", function="count")

i.e. the same call, but different syntax.

I might just go open a ticket for this soon anyway to get it recorded
properly, and at the very least release the code as a standalone module
- it's written so it can work like that anyway.

Andrew

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[A Mele]

I have followed the discussions about denormalization and I really
would like to see this in trunk. I agree with Rusell, the syntax
should be more like Django's query syntax, but I think this kind of
field is very useful for a lot of projects.


A Mele



On 27 oct, 13:26, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> Russell Keith-Magee wrote:
> > I do have a slight reservation regarding the API you are proposing.
> > The fields you have proposed (MirrorField and AggregateField) capture
> > the obvious use cases, but the syntax don't feel consistent with the
> > rest of the Django API. In particular, AggregateField seems to be
> > introducing a whole new query syntax that is completely independent of
> > the aggregation syntax that is due for inclusion in v1.1.
>
> > I would prefer to see a syntax that demonstrated slightly better
> > integration with the Django query syntax. If it isn't possible to use
> > QuerySet itself as a way to develop the underlying query, then I would
> > expect to see a syntax that at the very least retained the flavour of
> > the Django query syntax rather than inventing "fk:" style operators
> > that have no real analog.
>
> Indeed; I was far happier with the queryset syntax for AggregateField I
> originally proposed, but I couldn't get it to work - it just won't go
> inside class bodies without far, far too much hackery. The MirrorField
> API I like more; I have an alternate version where you simply pass it
> the model name and column name, but that then has issues if you have
> more than one FK to the same model.
>
> The AggregateField proposal is just that, and I really squirmed at the
> idea of "all:" and "fk:", but it got it working pretty quickly. One
> possibility for more clean integration is an approach more like
> querysets, so you could have something like
>
> photos_count = models.AggregateField(Photo.view.filter(user_self=True))
>
> The main problem with anything like this (and indeed with my attempt to
> implement this with plain old querysets) is that, as a field, you exist
> only on a class, and so when you get a signal saying a model has been
> updated, it's somewhat difficult to determine which instances of
> yourself to update - you can't loop through them all testing, since
> that's hilariously inefficient. That's why I limited the querying to
> only fk: and all: types, since detecting these is far more efficient.
>
> I'd love to see any proposal for how Aggregates should look, as my
> current incarnation really isn't too django-ish. I personally like
> MirrorFields (now on their third name, more welcomed) as I have them
> now, but then I would, since I invented them for my own use...
>
> Andrew
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Andrew Godwin]

Russell Keith-Magee wrote:
> I do have a slight reservation regarding the API you are proposing.
> The fields you have proposed (MirrorField and AggregateField) capture
> the obvious use cases, but the syntax don't feel consistent with the
> rest of the Django API. In particular, AggregateField seems to be
> introducing a whole new query syntax that is completely independent of
> the aggregation syntax that is due for inclusion in v1.1.
>
> I would prefer to see a syntax that demonstrated slightly better
> integration with the Django query syntax. If it isn't possible to use
> QuerySet itself as a way to develop the underlying query, then I would
> expect to see a syntax that at the very least retained the flavour of
> the Django query syntax rather than inventing "fk:" style operators
> that have no real analog.
>   

Indeed; I was far happier with the queryset syntax for AggregateField I
originally proposed, but I couldn't get it to work - it just won't go
inside class bodies without far, far too much hackery. The MirrorField
API I like more; I have an alternate version where you simply pass it
the model name and column name, but that then has issues if you have
more than one FK to the same model.

The AggregateField proposal is just that, and I really squirmed at the
idea of "all:" and "fk:", but it got it working pretty quickly. One
possibility for more clean integration is an approach more like
querysets, so you could have something like

photos_count = models.AggregateField(Photo.view.filter(user_self=True))

The main problem with anything like this (and indeed with my attempt to
implement this with plain old querysets) is that, as a field, you exist
only on a class, and so when you get a signal saying a model has been
updated, it's somewhat difficult to determine which instances of
yourself to update - you can't loop through them all testing, since
that's hilariously inefficient. That's why I limited the querying to
only fk: and all: types, since detecting these is far more efficient.

I'd love to see any proposal for how Aggregates should look, as my
current incarnation really isn't too django-ish. I personally like
MirrorFields (now on their third name, more welcomed) as I have them
now, but then I would, since I invented them for my own use...

Andrew

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation Magic, Round Two

[Russell Keith-Magee]

On Mon, Oct 27, 2008 at 8:05 AM, Andrew Godwin <[EMAIL PROTECTED]> wrote:
>
> class User(models.Model):
>username = models.CharField(max_length=255)
>email = models.TextField()
>num_photos = models.AggregateField("Photo", "fk:user", "count")
>num_all_photos = models.AggregateField("Photo", "all:", "count")
>
>def __unicode__(self):
>return "%s <%s>" % (self.username, self.email)
>
> class Photo(models.Model):
>caption = models.TextField()
>user = models.ForeignKey(User, related_name="photos")
>user_username = models.MirrorField("user", "username")
>
>def __unicode__(self):
>return "%s by %s" % (self.caption, self.user_username)
>
...
> So, what does everyone think? Useful, or just too much of an edge case?
> Reasonable patch, or badly implemented?

First off, details and implementation notwithstanding, I'm +1 for
adding denormalization fields of this sort. Apologies for not getting
involved during the first pass of this discussion.

I do have a slight reservation regarding the API you are proposing.
The fields you have proposed (MirrorField and AggregateField) capture
the obvious use cases, but the syntax don't feel consistent with the
rest of the Django API. In particular, AggregateField seems to be
introducing a whole new query syntax that is completely independent of
the aggregation syntax that is due for inclusion in v1.1.

I would prefer to see a syntax that demonstrated slightly better
integration with the Django query syntax. If it isn't possible to use
QuerySet itself as a way to develop the underlying query, then I would
expect to see a syntax that at the very least retained the flavour of
the Django query syntax rather than inventing "fk:" style operators
that have no real analog.

MirrorField is essentially a query that returns a related object;
AggregateField is a query that returns an aggregate of related
objects. Looking longer term, this sort of approach also has the
potential to be integrated into database views, since defining a view
requires similar querying capabilities.

Unfortunately, I haven't given this enough thought to be able to
suggest an alternate syntax that will do the job. At this point, I
just know that I would rather avoid introducing 2 different syntaxes
for aggregates. I have a hazy vision in my head of the sort of thing
that I think could work; I'll try to get it to coalesce a little and
get back to you.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---