Re: Making QuerySets composable

[Anssi Kääriäinen]

We have room for some improvement here. It should be possible to add
subqueries to the current query. This would be especially useful for
adding aggregates.

For aggregates we need this feature because the current API doesn't
work for anything but simple cases. For example
Book.objects.annotate(authors_cnt=Count('authors'),
suppliers_cnt=Count('suppliers')) will currently produce two joins,
one to authors and one to suppliers. Thus, if a book has three authors
and two suppliers, it will have both counts as six. Solving this
problem (taking in account .filter() calls, too) is extremely complex.

If the user could instead tell the ORM to add the counts in
subqueries, the ORM would have much easier job of generating a correct
query.

For the "add single aggregate case" it might be sufficient to add a
strategy flag to aggregates. So,
Book.objects.annotate(authors_cnt=Count('authors',
strategy=models.SUBQUERY)) is a possible API.

If you want multiple aggregates in a single subquery we likely need
something more explicit. Maybe something like
Book.objects.add_related(
authors_subq=models.AggregateSubquery(
Author.objects.annotate(cnt=Count('pk'), avg_age=Avg('age')))
This would add authors_subq__cnt and authors_subq__avg_age
automatically to the outer query as annotations.

The add_related() method would come handy in other cases, too.
Consider joining a translation for currently active language:
Book.objects.add_related(
current_translation=models.ModelRelation('translations',
only=Q(lang=get_current_language()))

You could also attach raw SQL in the same way:
Room.objects.add_related(
current_reservation=models.RawSQLRelation(
"select * from reservations where start_date <=
now()::date and end_date >= now()::date",
on_condition=Q(pk=F('current_reservation__room_id')),
model=Reservation)
))

We could later on add support for more esoteric concepts like lateral
joins or materializing the subquery into a temporary table.

I believe the current API for aggregates and multivalued relations has
come to a dead end. Simpler queries work well enough with the current
API. For more complex queries the API isn't easy to understand, and it
is extremely hard to translate complex ORM queries to SQL. Consider
Book.objects.filter(
Q(authors__height__gte=180), ~Q(authors__age__gte=50)
).annotate(author_cnt=Count('authors'))

This query is *extremely* hard to translate to SQL. Just the first
filter case is something nasty, the Q-object doesn't mean "keep all
rows that has author's age less than 50 and height greater than 180".
It means "a book must have at least one author taller than 180. *All*
of such authors for the book must have age less than 50 or the age
must be null". Still, we need to generate a count over such authors
for the book. And this example doesn't even reveal the full complexity
of the problem.

In summary, I vote +1 for the ability to compose querysets. The first
issue to tackle is to come up with a good extensible API for this.

 - Anssi

On Fri, Jan 22, 2016 at 6:44 PM, charettes  wrote:
> Hi Patryk,
>
>> Currently we have Prefetch objects that kind of solve this problem for M2M
>> relations but perhaps it would be nice to also be able to use QuerySets in
>> select_related() or even in filter(). I don't think Prefetch objects are
>> best
>> suited for that and I personally find having to instantiate them
>> explicitly
>> kind of ugly.
>
> From what I understand you're not a fan of Prefetch objects but I think most
> of your problems can be solved by using them and custom managers as the
> formers
> also support foreign keys[1].
>
> class AuthorDisplayManager(models.Manager):
> def get_queryset(self):
> queryset = super().get_queryset()
> return queryset.filter(
> visible=True,
> ).annotate(
> num_books=Count('books'),
> num_awards=Count('awards'),
> ).prefetch_related(
> 'foo',
> )
>
> class Author(models.Model):
> ...
> for_display = AuthorDisplayManager()
>
>
> class BookDisplayManager(models.Manager):
> def get_queryset(self):
> queryset = super().get_queryset()
> return queryset.filter(
> visible=True,
> ).prefetch_related(
> Prefetch(
> 'author',
> Author.for_display.all(),
> )
> )
>
>
> class Book(models.Model):
> ...
> for_display = BookDisplayManager()
>
> Simon
>
> [1] https://code.djangoproject.com/ticket/17003
>
> Le vendredi 22 janvier 2016 05:25:24 UTC-5, Patryk Zawadzki a écrit :
>>
>> Hi,
>>
>> Currently the way QuerySets work makes it hard to reuse code. Consider an
>> example where I have template tags to render an author and a book (that in
>> turn displays author). I don't want my template code to execute any
>> additional queries per row displayed so I am forced to prefetch ev

Re: does anyone use contrib.auth's "test models"? (CustomUser and ExtensionUser)

[Markus Holtermann]

Thanks Simon and Tim,

I'd be ok with moving them into the tests in 1.10, but would give people a 
bit more time to see and respond to this discussion. Maybe until Feb 4th, 
that would be 2 weeks.

I haven't seen anybody actually using the models nor do I use them myself.

/Markus

On Thursday, January 21, 2016 at 1:38:44 PM UTC+11, Tim Graham wrote:
>
> As Simon noted in a ticket [1], "Since the introduction of contrib 
> application migrations in 1.8 (#22170 
> ) the ​documented custom 
> user test models 
> 
>  
> table are not created anymore because they are registered to the 'auth' 
> labeled application but are not part of its shipped migrations. Based on 
> the fact this was not reported before and that the only uses of 
> ExtensionUser I can find is ​either from a vendored version of Django or 
> a copy-pasta of the documentation suggested usage 
> 
>  
> I think we can safely assume it's not commonly used among third-party 
> application who claim to support custom user models."
>
>
> Any objection to removing those models from documentation in 1.8 and 1.9 
> and removing the models themselves in 1.10? (Actually, the models would be 
> moved into the test suite instead of shipped in 
> django/contrib/auth/tests/custom_user.py -- this is done in [2].)
>
>
> [1] https://code.djangoproject.com/ticket/26089
>
> [2] https://github.com/django/django/pull/5975
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8cb33a52-bf56-4cd1-88d3-effafe06e335%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Deprecate choices argument in Select.render

[James Pic]

Felt free, thanks for answering.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALC3Kaf8R08bn9izEiraJeKeAeg6c%3D896tGctdx89N_3kYVaKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Deprecate choices argument in Select.render

[Tim Graham]

Please see https://code.djangoproject.com/ticket/25731 for some discussion. 
Feel free to offer a patch so we can assess if it's feasible.

On Sunday, January 24, 2016 at 2:57:04 PM UTC-5, is_null wrote:
>
> Hi all,
>
> In Select widgets, as well as many other (including MultipleHiddenInput, 
> where it's not used) rendering methods all have a choices=() kwarg and use 
> a chain(self.choices, choices) when rendering.
>
> Maybe I'm missing something, but it seems like this has just been sitting 
> there since 2006 and all the sensible use cases for this have disappeared: 
> why would anyone do the effort of adding a widget-specific override in 
> Form.as_widget to add choices to rendering that won't even validate ?
>
> Can we engage deprecation on the choices argument in render methods and 
> rely on self.choices only or am I missing something ?
>
> If so, do you know if any work on this has been started ?
>
> Best
>
> James
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/480c6bde-7e4a-4b91-b7cd-43bdb9ea46fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Deprecate choices argument in Select.render

[James Pic]

Hi all,

In Select widgets, as well as many other (including MultipleHiddenInput,
where it's not used) rendering methods all have a choices=() kwarg and use
a chain(self.choices, choices) when rendering.

Maybe I'm missing something, but it seems like this has just been sitting
there since 2006 and all the sensible use cases for this have disappeared:
why would anyone do the effort of adding a widget-specific override in
Form.as_widget to add choices to rendering that won't even validate ?

Can we engage deprecation on the choices argument in render methods and
rely on self.choices only or am I missing something ?

If so, do you know if any work on this has been started ?

Best

James

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALC3KacRiBEF3_0otH1jVw3Hg08SeyzberBv0p3bPWge4oTEAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.