Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method
My take on this is that .extra and .raw are 'hacks' that only exist because the existing functions of the ORM are too limited. There are a few changes coming to extend the utility of .annotate (and .aggregate), along with the Lookup and Transform APIs, that should solve 90% of use cases that .extra is currently used for. Therefore I don't think that extending the utility of .extra is the right way forward. Even with the coming changes mentioned above, there are still limitations, and forcing parts of a query into a subquery is not really supported. Trying to use a negated Q() with m2m joins is an example I believe, and so is aggregating over m2m joins (someone will correct me if I'm wrong - I'm sure). I think time and effort would be better spent on designing a solution to the above issues. Being able to force a particular query into a subquery will be extremely useful, and will solve the bugs causing you to ask for a .join implementation. I totally agree that there is a need for join handling, but I think it's better left as an internal django function that does the correct thing when needed. Regards, Josh On Tuesday, 11 March 2014 04:37:09 UTC+11, Matthieu Rigal wrote: > > Sorry to reopen an old thread, but this was is indicated to do in the > ticket doc. > > I think actually extra is a very helpful function and that join is a > missing part of it. I'm thinking especially about the admin part, where one > sometime wants to add count of foreign keys to the view. Since the > aggregate is so unperformant and returns bad results if you want more than > one count, but you also want to keep the filtering and ordering features of > the admin interface, there is no better solution than join as extra. > > One example : LEFT OUTER JOIN (SELECT COUNT(*) AS "devices_count", > "accounts_venuedevice"."venue_id" FROM "accounts_venuedevice" WHERE > "accounts_venuedevice"."enabled"=true GROUP BY > "accounts_venuedevice"."venue_id" ) AS "enabled_devices" ON ( > "accounts_venue"."id" = "enabled_devices"."venue_id") > > raw() is not a good option for this case and the classic ORM neither is an > option because of wrong and slow queries... > > Best, > Matthieu > > On Saturday, March 12, 2011 6:06:36 PM UTC+1, bendavis78 wrote: >> >> Even if it is a kludge, it still accomplishes something that .raw() >> cannot (as Dan put forth). I think deprecating it in favor of raw doesn't >> make much sense, since they are two different things. >> On Mar 9, 2011 4:06 PM, "Dan Watson" wrote: >> > >> > >> > On Tuesday, March 8, 2011 6:16:26 PM UTC-5, Russell Keith-Magee wrote: >> >> >> >> On Wed, Mar 9, 2011 at 7:09 AM, Jacob Kaplan-Moss >> >> >> wrote: >> >> > On Tue, Mar 8, 2011 at 9:35 PM, bendavis78 >> wrote: >> >> >> I'd like to start a discussion on this since russelm closed the >> >> >> issue. There are a few other people that believe the issue should be >> >> >> left open. I've been using this patch for nearly two years, and have >> >> >> found it to be useful in several different cases. I disagree that >> >> >> the .raw() functionality is a sufficient alternative, as it is not >> >> >> possible to modify an existing queryset using .raw(). For example, >> >> >> if I have a function that accepts a queryset, I want to be able to >> >> >> modify that queryset by giving it a extra info for the JOIN and >> SELECT >> >> >> clauses. >> >> > >> >> > .extra() was a kludge that existed because .raw() didn't. Frankly, >> I'm >> >> > considering deprecating and removing .extra() entirely: >> >> >> >> Yes. Yes Yes Yes. Yes. Oh, and Yes. +1. >> >> >> >> > I've rarely >> >> > seen a case where using it didn't come back to cause problems in the >> >> > future. I'm certainly going to be a strong -1 on adding any more >> >> > "features" to .extra(). >> >> >> >> Agreed. From an engineering perspective, extra() is the single most >> >> fragile part of the ORM. Dumping extra would make me extraordinarily >> >> happy. >> >> >> > It's also incredibly useful for certain situations. Two that come >> > immediately to mind are augmenting objects returned with an extra field >> via >> > custom SQL (a subselect for instance), and using custom SQL (e.g. >> to_tsquery >> > for full-text searches in Postgres) in the WHERE clause. Both *could* >> be >> > accomplished using .raw() if you don't mind giving up .filter(), >> > .order_by(), etc. But it would be a shame to throw out everything a >> queryset >> > gives you just because you need hooks into the SQL it generates. >> > >> > If you're seriously considering deprecating .extra(), I think it would >> be >> > beneficial to put together some use cases (such as those above) that >> could >> > be covered in a less-fragile way. >> > >> > Dan >> > >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Django developers" group. >> > To post to this group, send email to django-d...@googlegroups.com. >> > To unsu
Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method
Sorry to reopen an old thread, but this was is indicated to do in the ticket doc. I think actually extra is a very helpful function and that join is a missing part of it. I'm thinking especially about the admin part, where one sometime wants to add count of foreign keys to the view. Since the aggregate is so unperformant and returns bad results if you want more than one count, but you also want to keep the filtering and ordering features of the admin interface, there is no better solution than join as extra. One example : LEFT OUTER JOIN (SELECT COUNT(*) AS "devices_count", "accounts_venuedevice"."venue_id" FROM "accounts_venuedevice" WHERE "accounts_venuedevice"."enabled"=true GROUP BY "accounts_venuedevice"."venue_id" ) AS "enabled_devices" ON ( "accounts_venue"."id" = "enabled_devices"."venue_id") raw() is not a good option for this case and the classic ORM neither is an option because of wrong and slow queries... Best, Matthieu On Saturday, March 12, 2011 6:06:36 PM UTC+1, bendavis78 wrote: > > Even if it is a kludge, it still accomplishes something that .raw() cannot > (as Dan put forth). I think deprecating it in favor of raw doesn't make > much sense, since they are two different things. > On Mar 9, 2011 4:06 PM, "Dan Watson" > > wrote: > > > > > > On Tuesday, March 8, 2011 6:16:26 PM UTC-5, Russell Keith-Magee wrote: > >> > >> On Wed, Mar 9, 2011 at 7:09 AM, Jacob Kaplan-Moss > >> wrote: > >> > On Tue, Mar 8, 2011 at 9:35 PM, bendavis78 > wrote: > >> >> I'd like to start a discussion on this since russelm closed the > >> >> issue. There are a few other people that believe the issue should be > >> >> left open. I've been using this patch for nearly two years, and have > >> >> found it to be useful in several different cases. I disagree that > >> >> the .raw() functionality is a sufficient alternative, as it is not > >> >> possible to modify an existing queryset using .raw(). For example, > >> >> if I have a function that accepts a queryset, I want to be able to > >> >> modify that queryset by giving it a extra info for the JOIN and > SELECT > >> >> clauses. > >> > > >> > .extra() was a kludge that existed because .raw() didn't. Frankly, I'm > >> > considering deprecating and removing .extra() entirely: > >> > >> Yes. Yes Yes Yes. Yes. Oh, and Yes. +1. > >> > >> > I've rarely > >> > seen a case where using it didn't come back to cause problems in the > >> > future. I'm certainly going to be a strong -1 on adding any more > >> > "features" to .extra(). > >> > >> Agreed. From an engineering perspective, extra() is the single most > >> fragile part of the ORM. Dumping extra would make me extraordinarily > >> happy. > >> > > It's also incredibly useful for certain situations. Two that come > > immediately to mind are augmenting objects returned with an extra field > via > > custom SQL (a subselect for instance), and using custom SQL (e.g. > to_tsquery > > for full-text searches in Postgres) in the WHERE clause. Both *could* be > > accomplished using .raw() if you don't mind giving up .filter(), > > .order_by(), etc. But it would be a shame to throw out everything a > queryset > > gives you just because you need hooks into the SQL it generates. > > > > If you're seriously considering deprecating .extra(), I think it would > be > > beneficial to put together some use cases (such as those above) that > could > > be covered in a less-fragile way. > > > > Dan > > > >> > > > > -- > > You received this message because you are subscribed to the Google > Groups "Django developers" group. > > To post to this group, send email to > > django-d...@googlegroups.com > . > > To unsubscribe from this group, send email to > django-develop...@googlegroups.com . > > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1c2227b1-1d6e-482c-b73b-786c6189656f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method
Even if it is a kludge, it still accomplishes something that .raw() cannot (as Dan put forth). I think deprecating it in favor of raw doesn't make much sense, since they are two different things. On Mar 9, 2011 4:06 PM, "Dan Watson" wrote: > > > On Tuesday, March 8, 2011 6:16:26 PM UTC-5, Russell Keith-Magee wrote: >> >> On Wed, Mar 9, 2011 at 7:09 AM, Jacob Kaplan-Moss >> wrote: >> > On Tue, Mar 8, 2011 at 9:35 PM, bendavis78 wrote: >> >> I'd like to start a discussion on this since russelm closed the >> >> issue. There are a few other people that believe the issue should be >> >> left open. I've been using this patch for nearly two years, and have >> >> found it to be useful in several different cases. I disagree that >> >> the .raw() functionality is a sufficient alternative, as it is not >> >> possible to modify an existing queryset using .raw(). For example, >> >> if I have a function that accepts a queryset, I want to be able to >> >> modify that queryset by giving it a extra info for the JOIN and SELECT >> >> clauses. >> > >> > .extra() was a kludge that existed because .raw() didn't. Frankly, I'm >> > considering deprecating and removing .extra() entirely: >> >> Yes. Yes Yes Yes. Yes. Oh, and Yes. +1. >> >> > I've rarely >> > seen a case where using it didn't come back to cause problems in the >> > future. I'm certainly going to be a strong -1 on adding any more >> > "features" to .extra(). >> >> Agreed. From an engineering perspective, extra() is the single most >> fragile part of the ORM. Dumping extra would make me extraordinarily >> happy. >> > It's also incredibly useful for certain situations. Two that come > immediately to mind are augmenting objects returned with an extra field via > custom SQL (a subselect for instance), and using custom SQL (e.g. to_tsquery > for full-text searches in Postgres) in the WHERE clause. Both *could* be > accomplished using .raw() if you don't mind giving up .filter(), > .order_by(), etc. But it would be a shame to throw out everything a queryset > gives you just because you need hooks into the SQL it generates. > > If you're seriously considering deprecating .extra(), I think it would be > beneficial to put together some use cases (such as those above) that could > be covered in a less-fragile way. > > Dan > >> > > -- > You received this message because you are subscribed to the Google Groups "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method
On Tuesday, March 8, 2011 6:16:26 PM UTC-5, Russell Keith-Magee wrote: > > On Wed, Mar 9, 2011 at 7:09 AM, Jacob Kaplan-Moss > wrote: > > On Tue, Mar 8, 2011 at 9:35 PM, bendavis78 wrote: > >> I'd like to start a discussion on this since russelm closed the > >> issue. There are a few other people that believe the issue should be > >> left open. I've been using this patch for nearly two years, and have > >> found it to be useful in several different cases. I disagree that > >> the .raw() functionality is a sufficient alternative, as it is not > >> possible to modify an existing queryset using .raw(). For example, > >> if I have a function that accepts a queryset, I want to be able to > >> modify that queryset by giving it a extra info for the JOIN and SELECT > >> clauses. > > > > .extra() was a kludge that existed because .raw() didn't. Frankly, I'm > > considering deprecating and removing .extra() entirely: > > Yes. Yes Yes Yes. Yes. Oh, and Yes. +1. > > > I've rarely > > seen a case where using it didn't come back to cause problems in the > > future. I'm certainly going to be a strong -1 on adding any more > > "features" to .extra(). > > Agreed. From an engineering perspective, extra() is the single most > fragile part of the ORM. Dumping extra would make me extraordinarily > happy. > It's also incredibly useful for certain situations. Two that come immediately to mind are augmenting objects returned with an extra field via custom SQL (a subselect for instance), and using custom SQL (e.g. to_tsquery for full-text searches in Postgres) in the WHERE clause. Both *could* be accomplished using .raw() if you don't mind giving up .filter(), .order_by(), etc. But it would be a shame to throw out everything a queryset gives you just because you need hooks into the SQL it generates. If you're seriously considering deprecating .extra(), I think it would be beneficial to put together some use cases (such as those above) that could be covered in a less-fragile way. Dan > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method
On Wed, Mar 9, 2011 at 7:09 AM, Jacob Kaplan-Moss wrote: > On Tue, Mar 8, 2011 at 9:35 PM, bendavis78 wrote: >> I'd like to start a discussion on this since russelm closed the >> issue. There are a few other people that believe the issue should be >> left open. I've been using this patch for nearly two years, and have >> found it to be useful in several different cases. I disagree that >> the .raw() functionality is a sufficient alternative, as it is not >> possible to modify an existing queryset using .raw(). For example, >> if I have a function that accepts a queryset, I want to be able to >> modify that queryset by giving it a extra info for the JOIN and SELECT >> clauses. > > .extra() was a kludge that existed because .raw() didn't. Frankly, I'm > considering deprecating and removing .extra() entirely: Yes. Yes Yes Yes. Yes. Oh, and Yes. +1. > I've rarely > seen a case where using it didn't come back to cause problems in the > future. I'm certainly going to be a strong -1 on adding any more > "features" to .extra(). Agreed. From an engineering perspective, extra() is the single most fragile part of the ORM. Dumping extra would make me extraordinarily happy. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method
On Tue, Mar 8, 2011 at 9:35 PM, bendavis78 wrote: > I'd like to start a discussion on this since russelm closed the > issue. There are a few other people that believe the issue should be > left open. I've been using this patch for nearly two years, and have > found it to be useful in several different cases. I disagree that > the .raw() functionality is a sufficient alternative, as it is not > possible to modify an existing queryset using .raw(). For example, > if I have a function that accepts a queryset, I want to be able to > modify that queryset by giving it a extra info for the JOIN and SELECT > clauses. .extra() was a kludge that existed because .raw() didn't. Frankly, I'm considering deprecating and removing .extra() entirely: I've rarely seen a case where using it didn't come back to cause problems in the future. I'm certainly going to be a strong -1 on adding any more "features" to .extra(). Remember, Django's ORM tries to hit an 80-90% point, but then you're *expected* to fall back to raw SQL past that point. Falling back to raw SQL when it's easier is a feature, not a bug. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
