I like this idea, too.
The work I and Loic have been doing has been about transforms that can
accept arguments. I don't see why the two styles couldn't be mixed
directly.
.filter(Q.user.profile.last_login.date(timezone='Europe/Helsinki') <
'2015-01-01')
.filter(Q.user.last_name.collate('fi').unaccent() == 'Kaariainen')
OTOH if you want to use expressions directly, that should be OK, too
(if you don't mind the non-optimal readability):
.filter(Exact(unaccent(collate(Q.user.last_name, 'fi'))), 'Kaariainen'))
All in all, +1 to working towards these goals. I'm not yet convinced
we need all this in Django core directly. Maybe we should first try
these ideas in 3rd party apps, and then pick a proven solution into
Django core?
- Anssi
On Mon, Aug 17, 2015 at 3:33 AM, Josh Smeaton <josh.smea...@gmail.com> wrote:
> Hi Alexey,
>
> I find this approach really interesting, and I don't mind the API you've
> created at all. Even if this doesn't make it into Django, I think there's
> enough utility in your library that other people would want to use it. I'm
> not going to comment on specifics just yet like method names, because I
> figure that if we can agree on an approach, the bike-shedding can come
> later.
>
> So first things first. I think we can all, or at least mostly, agree that
> having an alternative syntax for building filters is a good idea. String
> based lookups are cool, and won't go away, but sometimes you want to have
> greater control over what kind of filter you want to build without having to
> register global lookups and transforms. I see similarities with
> prefetch_related. You can provide a string based field lookup, or use a
> Prefetch() object to give users more control over the behaviour of the
> prefetch.
>
> I've been working on a patch (https://github.com/django/django/pull/5090) to
> unify Transforms and Expressions (Func specifically). The work behind that
> can be (and should be) extended to include Lookups -- making them fully
> fledged expression objects. That work will need to happen regardless. Once
> this patch is finished and lands, then we should be able to extend .filter()
> and .exclude() to allow expressions to be composed as filters:
>
> GameSession.objects.filter(GreaterThan('user__profile__last_login_date',
> yesterday))
> # or, if last_login was a datetime, and we wanted to compare the date part
> only
> GameSession.objects.filter(Equal(Date('user__profile__last_login'),
> datetime.now().date))
> # or if we wanted to implement __gt__ and __eq__ etc:
> GameSession.objects.filter(Date('user__profile__last_login') ==
> datetime.now().date))
>
> Loic and Anssi have also been working on alternative syntaxes
> (https://github.com/django/django/pull/4953):
>
> GameSession.objects.filter(E('user__profile__last_login').date() ==
> datetime.now().date)
>
>
> Article.objects.filter(E('headline').collate('fi') == 'Article1')
>
> Both of these approaches still suffer from "the string problem" that you're
> trying to address, but minimises the final component by extracting the
> lookup and transform components as objects instead. So I think your idea
> here could nicely coexist:
>
> GameSession.objects.filter(Equal(Date(Q.user.profile.last_login),
> datetime.now().date))
> GameSession.objects.filter(E(Q.user.profile.last_login).date() ==
> datetime.now().date)
>
> Or, even building this into F expressions rather than Q expressions:
>
> GameSession.objects.filter(Equal(Date(F.user.profile.last_login),
> datetime.now().date))
> GameSession.objects.filter(E(F.user.profile.last_login).date() ==
> datetime.now().date)
>
> I think it may look better with F objects, considering they are Field
> references, and since the Lookups (Equal/GTE) parts accept F expressions
> anyway. I'm not too concerned about this particular detail though.
>
> A DEP is probably the right way to go here but I wonder if we should expand
> the scope to include alternative filter syntax as a whole
> (expressions/callable transforms) as well as the dot field reference
> notation you've proposed above. Then we can consider how all these things
> might work together, and clearly document why we've gone one way and not
> another. Obviously, alternatives can exist outside of core where the API is
> available.
>
> I'll be happy to work as the shepherd if needed. But I'd also like some
> input from Loic and Anssi especially, as well as others in the core team
> with some interest in the ORM.
>
> Regards,
>
>
> On Sunday, 16 August 2015 23:18:26 UTC+10, Alexey Zankevich wrote:
>>
>> Hi all,
>>
>> This topic is related to the current ORM query syntax with underscores.
>> There are lots of arguing related to it, anyway it has pros and cons.
>>
>> Let's take a concrete example of querying a model:
>>
>> >>>
>> >>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday)
>>
>>
>> Pros:
>>
>> 1. The syntax is easy to understand
>> 2. Can be extended with custom transforms and lookups
>>
>> However, there are several cons:
>>
>> 1. Long strings is hard to read, especially if we have fields with
>> underscores.
>> It's really easy to make a mistake by missing one:
>>
>> >>>
>> >>> GameSession.objects.filter(user_profile__last_login_date__gte=yesterday)
>>
>> Not easy to catch missing underscore between user and profile, is it? Even
>> though, it's not easy to say whether it should be "user_profile" attribute
>> or
>> user.profile foreign key.
>>
>> 2. Query strings can't be reused, thus the approach violates DRY
>> principle.
>> For example, we need to order results by last_login_date:
>>
>> >>>
>> >>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday)
>> >>> \
>> .order_by('user__profile__last_login_date')
>>
>> We can't keep user__profile_login_date as a variable as in the first part
>> of the
>> expression we use a keyword argument, meanwhile in the second part - just
>> a
>> string. And thus we just have to type query path twice.
>>
>> 3. Lookup names not natural to Python language and require to be
>> remembered or
>> looked up in documentation. For example, "__gte" or "__lte" lookups tend
>> to be
>> confused with "ge" and "le" due to similarity to methods "__ge__" and
>> "__le__".
>>
>> 4. Lookup keywords limited to a single argument only, very inconvenient
>> when
>> necessary to filter objects by range.
>>
>> I was thinking a lot trying to solve those issues, keeping in mind Django
>> approaches. Finally I came up with solution to extend Q objects with dot
>> expression syntax:
>>
>> >>> GameSession.objecs.filter(Q.user.profile.last_login_date >= yesterday)
>>
>> Q is a factory instance for old-style Q objects. Accessing attribute by
>> dot
>> returns a child factory, calling factory will instantiate old-style Q
>> object.
>>
>> >>> Q
>> <QFactory object at 0x7f407298ee10>
>>
>> >>> Q.user.profile
>> <QFactory object at 0x7f40765da310>
>>
>> >>> Q(user__name='Bob')
>> <Q: (AND: ('user__name', 'Bob'))>
>>
>> It overrides operators, so comparing factory with value returns a related
>> Q
>> object:
>>
>> >>> Q.user.name == 'Bob'
>> <Q: (AND: ('user__name', 'Bob'))>
>>
>> Factory has several helper functions for lookups which aren't related to
>> any
>> Python operators directly:
>>
>> >>> Q.user.name.icontains('Bob')
>> <Q: (AND: ('user__name__icontains', 'Bob'))>
>>
>> And helper to get query path as string, which requred by order_by or
>> select_related queryset methods:
>>
>> >>> Q.user.profile.last_login_date.get_path()
>> 'user__profile__last_login_date'
>>
>> You can check implementation and more examples here
>> https://github.com/Nepherhotep/django-orm-sugar
>>
>> How it solves issues:
>>
>> #1. Dots hard to confuse with underscores
>> #2. Query paths can be reused:
>>
>> >>> factory = Q.user.profile.last_login_date
>> >>> query = GameSession.objects.filter(factory >= yesterday)
>> >>> query = query.order_by(factory.get_path())
>>
>> #3. Not neccessary to remember most of lookup names and use comparison
>> operators
>> instead.
>> #4. Possible to use multiple keyword arguments:
>>
>> >>> Q.user.profile.last_login_date.in_range(from_date, to_date)
>> <Q: (AND: ('user__profile__last_login_date__lte', from_date),
>> ('user__profile__last_login_date__gte', to_date))>
>>
>>
>> This approach looked the best for me due to several reasons:
>>
>> 1. It's explicit - it doesn't do anything but generating appropriate Q
>> object.
>> The result of comparison can be saved as Q object variable.
>>
>> 2. It's short - variants with using model for that will look much longer,
>> when
>> joining two or more filters:
>>
>> >>> GameSession.objects.user.profile_last_login_date >= yesterday #
>> >>> awkward
>>
>> 3. Implementation will not require to change querset manager or model
>> classes
>>
>> 4. Will still allow to use filters and Q class in the old way:
>>
>> >>> q = Q(user__profile__last_login_date__gte=yesterday)
>>
>> or
>>
>> >>>
>> >>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday)
>>
>> I'd like to make it as a part of Django ORM syntax and it will not be hard
>> to
>> do, especially taking into account the library is already done and
>> working.
>> Anyway, I need your thought about the idea in general, as well as about
>> particular things like chosen method names - "get_path", "in_range" and
>> etc.
>> As next step I can create a ticket in the issue tracker, or prepare DEP
>> first.
>> In latter case I need to find a shepherd to work with.
>>
>> Best regards,
>> Alexey
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/db4003cb-f366-430c-92c1-8388ff35804b%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CALMtK1F3fgUcfCDyYKGKJFz9vyodwnVFY9KgWRpP_9n_XN_b7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
