Re: Complex query on inner join - case for __ne?

2019-04-15 Thread nm
I've only had a quick look at your problem, but looks like maybe this 
section of the Django documentation could be useful: 
https://docs.djangoproject.com/en/dev/topics/db/queries/#spanning-multi-valued-relationships


On Friday, 12 April 2019 15:10:59 UTC+2, Michael Thomas wrote:
>
> Hello everyone,
>
> I've run into what I believe is a limitation of the ORM that other people 
> must be dealing with somehow, but I can't seem to figure out a sensible 
> solution.
>
> I think it's easiest to describe the problem with code.
>
> For the following models:
>
> class Foo(models.Model):
> name = models.CharField(max_length=64)
>
>
> class Bar(models.Model):
> foo = models.ForeignKey(Foo, on_delete=models.CASCADE)
> attribute_1 = models.IntegerField()
> attribute_2 = models.IntegerField()
>
> I want to select all Foo() that have 1 or more bar with attribute_1 not 
> equal to 1, and attribute_2 equal to 2.
>
> Eg. SQL something like this:
>
> SELECT 
> "app_foo"."id", 
> "app_foo"."name" 
> FROM "app_foo" 
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> ) 
> WHERE (
> "app_bar"."attribute_1" <> 1 
> AND "app_bar"."attribute_2" = 2
> )
>
> However, here's what I end up with...
>
>
> print(Foo.objects.exclude(bar__attribute_1=1).filter(bar__attribute_2=2).query)
> SELECT 
> "app_foo"."id", 
> "app_foo"."name" 
> FROM "app_foo" 
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> ) 
> WHERE (
> NOT (
> "app_foo"."id" IN (
> SELECT 
> U1."foo_id" 
> FROM "app_bar" U1 
> WHERE U1."attribute_1" = 1
> )
> ) 
> AND "app_bar"."attribute_2" = 2
> )
>
> print(Foo.objects.filter(~Q(bar__attribute_1=1), bar__attribute_2=2).query)
> Exact same SQL output as above 
>
> Interestingly enough, a simple query for attribute_1=1 and attribute_2=2 
> works as expected, so it would be trivial to do this with a __ne operator 
> (if it existed), without any other changes to the ORM:
>
> print(Foo.objects.filter(bar__attribute_1=1, bar__attribute_2=2).query)
> SELECT 
> "app_foo"."id", 
> "app_foo"."name" 
> FROM "app_foo" 
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> ) 
> WHERE (
> "app_bar"."attribute_1" = 1 
> AND "app_bar"."attribute_2" = 2
> )
>
> Am I missing something here? How are other people tackling this?
>
> Kind Regards,
> Michael Thomas
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/8142abe8-1d14-4eff-aa6b-0f281ce14181%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Complex query on inner join - case for __ne?

2019-04-12 Thread Michael Thomas
Naimur: I am - that SQL was in response to  Aldian's question :)

On Fri, Apr 12, 2019 at 8:23 PM naimur rahman 
wrote:

> use ORM istead of SQL
>
> On Fri, Apr 12, 2019, 7:42 PM Michael Thomas <
> michael.thomas.s...@gmail.com> wrote:
>
>> SELECT
>> "app_foo"."id",
>> "app_foo"."name"
>> FROM "app_foo"
>> INNER JOIN "app_bar" ON (
>> "app_foo"."id" = "app_bar"."foo_id"
>> )
>> WHERE (
>> NOT (
>> "app_foo"."id" IN (
>> SELECT
>> U1."foo_id"
>> FROM "app_bar" U1
>> WHERE U1."attribute_1" = 1
>> )
>> )
>> AND "app_bar"."attribute_2" = 2
>> )
>>
>> On Fri, Apr 12, 2019 at 6:59 PM Aldian Fazrihady 
>> wrote:
>>
>>> What's the result of
>>>
>>> print(Foo.objects.exclude(bar__attribute=1).filter(
>>> bar__attribute_2=2).query)
>>>
>>> On Fri, 12 Apr 2019, 20:10 Michael Thomas, <
>>> michael.thomas.s...@gmail.com> wrote:
>>>
 Hello everyone,

 I've run into what I believe is a limitation of the ORM that other
 people must be dealing with somehow, but I can't seem to figure out a
 sensible solution.

 I think it's easiest to describe the problem with code.

 For the following models:

 class Foo(models.Model):
 name = models.CharField(max_length=64)


 class Bar(models.Model):
 foo = models.ForeignKey(Foo, on_delete=models.CASCADE)
 attribute_1 = models.IntegerField()
 attribute_2 = models.IntegerField()

 I want to select all Foo() that have 1 or more bar with attribute_1 not
 equal to 1, and attribute_2 equal to 2.

 Eg. SQL something like this:

 SELECT
 "app_foo"."id",
 "app_foo"."name"
 FROM "app_foo"
 INNER JOIN "app_bar" ON (
 "app_foo"."id" = "app_bar"."foo_id"
 )
 WHERE (
 "app_bar"."attribute_1" <> 1
 AND "app_bar"."attribute_2" = 2
 )

 However, here's what I end up with...


 print(Foo.objects.exclude(bar__attribute_1=1).filter(bar__attribute_2=2).query)
 SELECT
 "app_foo"."id",
 "app_foo"."name"
 FROM "app_foo"
 INNER JOIN "app_bar" ON (
 "app_foo"."id" = "app_bar"."foo_id"
 )
 WHERE (
 NOT (
 "app_foo"."id" IN (
 SELECT
 U1."foo_id"
 FROM "app_bar" U1
 WHERE U1."attribute_1" = 1
 )
 )
 AND "app_bar"."attribute_2" = 2
 )

 print(Foo.objects.filter(~Q(bar__attribute_1=1),
 bar__attribute_2=2).query)
 Exact same SQL output as above

 Interestingly enough, a simple query for attribute_1=1 and
 attribute_2=2 works as expected, so it would be trivial to do this with a
 __ne operator (if it existed), without any other changes to the ORM:

 print(Foo.objects.filter(bar__attribute_1=1, bar__attribute_2=2).query)
 SELECT
 "app_foo"."id",
 "app_foo"."name"
 FROM "app_foo"
 INNER JOIN "app_bar" ON (
 "app_foo"."id" = "app_bar"."foo_id"
 )
 WHERE (
 "app_bar"."attribute_1" = 1
 AND "app_bar"."attribute_2" = 2
 )

 Am I missing something here? How are other people tackling this?

 Kind Regards,
 Michael Thomas

 --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users+unsubscr...@googlegroups.com.
 To post to this group, send email to django-users@googlegroups.com.
 Visit this group at https://groups.google.com/group/django-users.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-users/d852fc10-5f5a-43e8-8dab-c796404867a8%40googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAN7EoAaXupYqNoPcHOcZ3OoA_07N--D4EPoEfZvkSWL%2BeLbD6A%40mail.gmail.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To 

Re: Complex query on inner join - case for __ne?

2019-04-12 Thread naimur rahman
use ORM istead of SQL

On Fri, Apr 12, 2019, 7:42 PM Michael Thomas 
wrote:

> SELECT
> "app_foo"."id",
> "app_foo"."name"
> FROM "app_foo"
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> )
> WHERE (
> NOT (
> "app_foo"."id" IN (
> SELECT
> U1."foo_id"
> FROM "app_bar" U1
> WHERE U1."attribute_1" = 1
> )
> )
> AND "app_bar"."attribute_2" = 2
> )
>
> On Fri, Apr 12, 2019 at 6:59 PM Aldian Fazrihady 
> wrote:
>
>> What's the result of
>>
>> print(Foo.objects.exclude(bar__attribute=1).filter(
>> bar__attribute_2=2).query)
>>
>> On Fri, 12 Apr 2019, 20:10 Michael Thomas, 
>> wrote:
>>
>>> Hello everyone,
>>>
>>> I've run into what I believe is a limitation of the ORM that other
>>> people must be dealing with somehow, but I can't seem to figure out a
>>> sensible solution.
>>>
>>> I think it's easiest to describe the problem with code.
>>>
>>> For the following models:
>>>
>>> class Foo(models.Model):
>>> name = models.CharField(max_length=64)
>>>
>>>
>>> class Bar(models.Model):
>>> foo = models.ForeignKey(Foo, on_delete=models.CASCADE)
>>> attribute_1 = models.IntegerField()
>>> attribute_2 = models.IntegerField()
>>>
>>> I want to select all Foo() that have 1 or more bar with attribute_1 not
>>> equal to 1, and attribute_2 equal to 2.
>>>
>>> Eg. SQL something like this:
>>>
>>> SELECT
>>> "app_foo"."id",
>>> "app_foo"."name"
>>> FROM "app_foo"
>>> INNER JOIN "app_bar" ON (
>>> "app_foo"."id" = "app_bar"."foo_id"
>>> )
>>> WHERE (
>>> "app_bar"."attribute_1" <> 1
>>> AND "app_bar"."attribute_2" = 2
>>> )
>>>
>>> However, here's what I end up with...
>>>
>>>
>>> print(Foo.objects.exclude(bar__attribute_1=1).filter(bar__attribute_2=2).query)
>>> SELECT
>>> "app_foo"."id",
>>> "app_foo"."name"
>>> FROM "app_foo"
>>> INNER JOIN "app_bar" ON (
>>> "app_foo"."id" = "app_bar"."foo_id"
>>> )
>>> WHERE (
>>> NOT (
>>> "app_foo"."id" IN (
>>> SELECT
>>> U1."foo_id"
>>> FROM "app_bar" U1
>>> WHERE U1."attribute_1" = 1
>>> )
>>> )
>>> AND "app_bar"."attribute_2" = 2
>>> )
>>>
>>> print(Foo.objects.filter(~Q(bar__attribute_1=1),
>>> bar__attribute_2=2).query)
>>> Exact same SQL output as above
>>>
>>> Interestingly enough, a simple query for attribute_1=1 and attribute_2=2
>>> works as expected, so it would be trivial to do this with a __ne operator
>>> (if it existed), without any other changes to the ORM:
>>>
>>> print(Foo.objects.filter(bar__attribute_1=1, bar__attribute_2=2).query)
>>> SELECT
>>> "app_foo"."id",
>>> "app_foo"."name"
>>> FROM "app_foo"
>>> INNER JOIN "app_bar" ON (
>>> "app_foo"."id" = "app_bar"."foo_id"
>>> )
>>> WHERE (
>>> "app_bar"."attribute_1" = 1
>>> AND "app_bar"."attribute_2" = 2
>>> )
>>>
>>> Am I missing something here? How are other people tackling this?
>>>
>>> Kind Regards,
>>> Michael Thomas
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/d852fc10-5f5a-43e8-8dab-c796404867a8%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAN7EoAaXupYqNoPcHOcZ3OoA_07N--D4EPoEfZvkSWL%2BeLbD6A%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> 

Re: Complex query on inner join - case for __ne?

2019-04-12 Thread Michael Thomas
SELECT
"app_foo"."id",
"app_foo"."name"
FROM "app_foo"
INNER JOIN "app_bar" ON (
"app_foo"."id" = "app_bar"."foo_id"
)
WHERE (
NOT (
"app_foo"."id" IN (
SELECT
U1."foo_id"
FROM "app_bar" U1
WHERE U1."attribute_1" = 1
)
)
AND "app_bar"."attribute_2" = 2
)

On Fri, Apr 12, 2019 at 6:59 PM Aldian Fazrihady  wrote:

> What's the result of
>
> print(Foo.objects.exclude(bar__attribute=1).filter(
> bar__attribute_2=2).query)
>
> On Fri, 12 Apr 2019, 20:10 Michael Thomas, 
> wrote:
>
>> Hello everyone,
>>
>> I've run into what I believe is a limitation of the ORM that other people
>> must be dealing with somehow, but I can't seem to figure out a sensible
>> solution.
>>
>> I think it's easiest to describe the problem with code.
>>
>> For the following models:
>>
>> class Foo(models.Model):
>> name = models.CharField(max_length=64)
>>
>>
>> class Bar(models.Model):
>> foo = models.ForeignKey(Foo, on_delete=models.CASCADE)
>> attribute_1 = models.IntegerField()
>> attribute_2 = models.IntegerField()
>>
>> I want to select all Foo() that have 1 or more bar with attribute_1 not
>> equal to 1, and attribute_2 equal to 2.
>>
>> Eg. SQL something like this:
>>
>> SELECT
>> "app_foo"."id",
>> "app_foo"."name"
>> FROM "app_foo"
>> INNER JOIN "app_bar" ON (
>> "app_foo"."id" = "app_bar"."foo_id"
>> )
>> WHERE (
>> "app_bar"."attribute_1" <> 1
>> AND "app_bar"."attribute_2" = 2
>> )
>>
>> However, here's what I end up with...
>>
>>
>> print(Foo.objects.exclude(bar__attribute_1=1).filter(bar__attribute_2=2).query)
>> SELECT
>> "app_foo"."id",
>> "app_foo"."name"
>> FROM "app_foo"
>> INNER JOIN "app_bar" ON (
>> "app_foo"."id" = "app_bar"."foo_id"
>> )
>> WHERE (
>> NOT (
>> "app_foo"."id" IN (
>> SELECT
>> U1."foo_id"
>> FROM "app_bar" U1
>> WHERE U1."attribute_1" = 1
>> )
>> )
>> AND "app_bar"."attribute_2" = 2
>> )
>>
>> print(Foo.objects.filter(~Q(bar__attribute_1=1),
>> bar__attribute_2=2).query)
>> Exact same SQL output as above
>>
>> Interestingly enough, a simple query for attribute_1=1 and attribute_2=2
>> works as expected, so it would be trivial to do this with a __ne operator
>> (if it existed), without any other changes to the ORM:
>>
>> print(Foo.objects.filter(bar__attribute_1=1, bar__attribute_2=2).query)
>> SELECT
>> "app_foo"."id",
>> "app_foo"."name"
>> FROM "app_foo"
>> INNER JOIN "app_bar" ON (
>> "app_foo"."id" = "app_bar"."foo_id"
>> )
>> WHERE (
>> "app_bar"."attribute_1" = 1
>> AND "app_bar"."attribute_2" = 2
>> )
>>
>> Am I missing something here? How are other people tackling this?
>>
>> Kind Regards,
>> Michael Thomas
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/d852fc10-5f5a-43e8-8dab-c796404867a8%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAN7EoAaXupYqNoPcHOcZ3OoA_07N--D4EPoEfZvkSWL%2BeLbD6A%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEdx1fpYmYHXX14iu3Enwc18Ojkk3%3DBfzTv3mGWVdf5BzZVv7w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Complex query on inner join - case for __ne?

2019-04-12 Thread Aldian Fazrihady
What's the result of

print(Foo.objects.exclude(bar__attribute=1).filter(
bar__attribute_2=2).query)

On Fri, 12 Apr 2019, 20:10 Michael Thomas, 
wrote:

> Hello everyone,
>
> I've run into what I believe is a limitation of the ORM that other people
> must be dealing with somehow, but I can't seem to figure out a sensible
> solution.
>
> I think it's easiest to describe the problem with code.
>
> For the following models:
>
> class Foo(models.Model):
> name = models.CharField(max_length=64)
>
>
> class Bar(models.Model):
> foo = models.ForeignKey(Foo, on_delete=models.CASCADE)
> attribute_1 = models.IntegerField()
> attribute_2 = models.IntegerField()
>
> I want to select all Foo() that have 1 or more bar with attribute_1 not
> equal to 1, and attribute_2 equal to 2.
>
> Eg. SQL something like this:
>
> SELECT
> "app_foo"."id",
> "app_foo"."name"
> FROM "app_foo"
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> )
> WHERE (
> "app_bar"."attribute_1" <> 1
> AND "app_bar"."attribute_2" = 2
> )
>
> However, here's what I end up with...
>
>
> print(Foo.objects.exclude(bar__attribute_1=1).filter(bar__attribute_2=2).query)
> SELECT
> "app_foo"."id",
> "app_foo"."name"
> FROM "app_foo"
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> )
> WHERE (
> NOT (
> "app_foo"."id" IN (
> SELECT
> U1."foo_id"
> FROM "app_bar" U1
> WHERE U1."attribute_1" = 1
> )
> )
> AND "app_bar"."attribute_2" = 2
> )
>
> print(Foo.objects.filter(~Q(bar__attribute_1=1), bar__attribute_2=2).query)
> Exact same SQL output as above
>
> Interestingly enough, a simple query for attribute_1=1 and attribute_2=2
> works as expected, so it would be trivial to do this with a __ne operator
> (if it existed), without any other changes to the ORM:
>
> print(Foo.objects.filter(bar__attribute_1=1, bar__attribute_2=2).query)
> SELECT
> "app_foo"."id",
> "app_foo"."name"
> FROM "app_foo"
> INNER JOIN "app_bar" ON (
> "app_foo"."id" = "app_bar"."foo_id"
> )
> WHERE (
> "app_bar"."attribute_1" = 1
> AND "app_bar"."attribute_2" = 2
> )
>
> Am I missing something here? How are other people tackling this?
>
> Kind Regards,
> Michael Thomas
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/d852fc10-5f5a-43e8-8dab-c796404867a8%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAN7EoAaXupYqNoPcHOcZ3OoA_07N--D4EPoEfZvkSWL%2BeLbD6A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Complex query, queryset.query.alias_map and query changes

2016-09-23 Thread Felipe Prenholato
Yeah, I know :\, but still maybe someone have a good idea :), lot of people
still using old Django versions.

I tried the expressions, the `F()` but it not worked well and I had no time
to debug why, but in next weeks I'll probably create a slug field to
compare directly.

I post here if have updates.

Thx Tim.

Felipe Prenholato.
Home page: http://devwithpassion.com | http://chronosbox.org/blog
GitHub: http://github.com/chronossc/ | Twitter: http://twitter.com/chronossc
LinkedIn: http://br.linkedin.com/in/felipeprenholato/

2016-09-23 13:36 GMT-03:00 Tim Graham :

> Not sure, this is using lots of private API and QuerySet.extra() which is
> discourage these days. I'm afraid you won't get much help for unsupported
> Django versions.
>
> After you upgrade to Django 1.9+, hopefully you can construct your query
> using the new expressions API!
>
> https://docs.djangoproject.com/en/stable/ref/models/expressions/
>
>
> On Friday, September 23, 2016 at 11:40:04 AM UTC-4, Felipe 'chronos'
> Prenholato wrote:
>>
>> Hello folks!
>>
>> I have a complex queryset here that do joins in at least 8 tables. The
>> query is fast, but has use of Postgresql `concat` method, and to make it
>> have correct aliases I'm using `qs.query.alias_map` where qs is my queryset
>> instance:
>>
>> def my_complex_queryset(self):
>> qs = self.filter(*lot_of_stuff_with_lot_of_joins)
>> alias_map = {
>> j.table_name: a for a, j in qs.query.alias_map.items()
>> if j.table_name in ['table_a', 'table_b']
>> }
>> concat_str = (
>> 'concat("{table_a}"."city", "{table_a}"."state") '
>> '!= concat("{table_b}"."city", "{table_b}"."state")'
>> ).format(
>> table_a=alias_map['table_a'],
>> table_b=alias_map['table_b']
>> )
>> qs = qs.extra(where=[concat_str])
>> return qs
>>
>> The problem is when the query is re-executed with some other table in the
>> joins because it will change the alias_map but won't run my method again
>> to  update the concat_str:
>>
>> my_qs = MyModel.objects.my_complex_queryset()
>> print my_qs
>> # now it will fail because alias changed:
>> print my_qs.filter(another_table_1__another_table_2__id__in=[1, 2, 3])
>>
>> The error I receive is:
>>
>> * ProgrammingError: invalid reference to FROM-clause entry for table
>> "table_a"*
>> *LINE 1: ... AND U0."some_id" IN (13361, 9820) AND (concat("table_a...*
>> * ^*
>> *HINT:  Perhaps you meant to reference the table alias "u9".*
>>
>> The great question is: Is possible to do the query with dynamic aliases
>> to be used in concat string? Maybe the `tables` argument to extra? I not
>> saw much doc about it :\.
>>
>> My Django version is 1.6.x, not because my choice but because software
>> here still a bit legacy about Django versions and we won't change until
>> January.
>>
>> Thanks for your attention!
>>
>> Felipe Prenholato.
>> Home page: http://devwithpassion.com | http://chronosbox.org/blog
>> GitHub: http://github.com/chronossc/ | Twitter: http://twitter.com/ch
>> ronossc
>> LinkedIn: http://br.linkedin.com/in/felipeprenholato/
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/77febd31-5b71-4ce2-a794-dab2f7c5e3e7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CADdUj3HxrWLJai5_wQaKWBKtBOi%2B9d_zZwCi0%3DPUm8yps_-gLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Complex query, queryset.query.alias_map and query changes

2016-09-23 Thread Tim Graham
Not sure, this is using lots of private API and QuerySet.extra() which is 
discourage these days. I'm afraid you won't get much help for unsupported 
Django versions.

After you upgrade to Django 1.9+, hopefully you can construct your query 
using the new expressions API!

https://docs.djangoproject.com/en/stable/ref/models/expressions/

On Friday, September 23, 2016 at 11:40:04 AM UTC-4, Felipe 'chronos' 
Prenholato wrote:
>
> Hello folks!
>
> I have a complex queryset here that do joins in at least 8 tables. The 
> query is fast, but has use of Postgresql `concat` method, and to make it 
> have correct aliases I'm using `qs.query.alias_map` where qs is my queryset 
> instance:
>
> def my_complex_queryset(self):
> qs = self.filter(*lot_of_stuff_with_lot_of_joins)
> alias_map = {
> j.table_name: a for a, j in qs.query.alias_map.items() 
> if j.table_name in ['table_a', 'table_b']
> }
> concat_str = (
> 'concat("{table_a}"."city", "{table_a}"."state") '
> '!= concat("{table_b}"."city", "{table_b}"."state")'
> ).format(
> table_a=alias_map['table_a'],
> table_b=alias_map['table_b']
> )
> qs = qs.extra(where=[concat_str])
> return qs
>
> The problem is when the query is re-executed with some other table in the 
> joins because it will change the alias_map but won't run my method again 
> to  update the concat_str:
>
> my_qs = MyModel.objects.my_complex_queryset()
> print my_qs
> # now it will fail because alias changed:
> print my_qs.filter(another_table_1__another_table_2__id__in=[1, 2, 3])
>
> The error I receive is:
>
> * ProgrammingError: invalid reference to FROM-clause entry for table 
> "table_a"*
> *LINE 1: ... AND U0."some_id" IN (13361, 9820) AND (concat("table_a...*
> * ^*
> *HINT:  Perhaps you meant to reference the table alias "u9".*
>
> The great question is: Is possible to do the query with dynamic aliases to 
> be used in concat string? Maybe the `tables` argument to extra? I not saw 
> much doc about it :\.
>
> My Django version is 1.6.x, not because my choice but because software 
> here still a bit legacy about Django versions and we won't change until 
> January.
>
> Thanks for your attention!
>
> Felipe Prenholato.
> Home page: http://devwithpassion.com | http://chronosbox.org/blog
> GitHub: http://github.com/chronossc/ | Twitter: 
> http://twitter.com/chronossc
> LinkedIn: http://br.linkedin.com/in/felipeprenholato/
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/77febd31-5b71-4ce2-a794-dab2f7c5e3e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Complex query reduction

2013-11-08 Thread Javier Guerra Giraldez
On Fri, Nov 8, 2013 at 1:44 AM, Robin St.Clair  wrote:
> The last time I checked the use of IN, all the records from the database in
> the query were brought back to the workstation, rather than being processed
> on the backend and only the results returned to the workstation.


Django ORM's __in operator tries to use SQL facilities, but depending
on the type of object passed, it could read the list of options and
pass it explicitly, or it could merge two SQL commands.  always check
the type of SQL commands generated!  (debug toolbar is your friend)

i haven't seen it doing a post-retrieve check in python, thought.

-- 
Javier

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFkDaoRLfeLi%3DWVu5f%3DcyCXRH0x5mr1fbDiV0OzW%2Bn-fpk6uHw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query reduction

2013-11-08 Thread François Schiettecatte
I am pretty sure the IN() performance issue in MySQL was fixed 5.5, and  
5.6/5.7 certainly don't have an issue with IN() whether you use a bunch of 
values or a subquery.

Cheers

François

On Nov 8, 2013, at 2:15 AM, akaariai  wrote:

> On Friday, November 8, 2013 8:44:09 AM UTC+2, Robin St.Clair wrote:
> Anssi
> 
> The last time I checked the use of IN, all the records from the database in 
> the query were brought back to the workstation, rather than being processed 
> on the backend and only the results returned to the workstation.
> 
> Have there been changes that carry out the entire query on the backend? What 
> has changed to cause you to prefer the use of the IN statement?
> 
> Django has had the ability to execute __in=qs in single query for a somewhat 
> long time (from Django 1.2 maybe?).
> 
> It is true that __in lookup against a large list of values is often a bad 
> choice even if you have the values at hand. Unfortunately this doesn't apply 
> to all databases, for example older versions of MySQL do not handle 
> subqueries well.
> 
>  - Anssi
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/f3428fda-e4be-4d75-8626-a95bd20f66c8%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Complex query reduction

2013-11-07 Thread akaariai
On Friday, November 8, 2013 8:44:09 AM UTC+2, Robin St.Clair wrote:
>
>  Anssi
>
> The last time I checked the use of IN, all the records from the database 
> in the query were brought back to the workstation, rather than being 
> processed on the backend and only the results returned to the workstation.
>
> Have there been changes that carry out the entire query on the backend? 
> What has changed to cause you to prefer the use of the IN statement?
>

Django has had the ability to execute __in=qs in single query for a 
somewhat long time (from Django 1.2 maybe?).

It is true that __in lookup against a large list of values is often a bad 
choice even if you have the values at hand. Unfortunately this doesn't 
apply to all databases, for example older versions of MySQL do not handle 
subqueries well.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/f3428fda-e4be-4d75-8626-a95bd20f66c8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query reduction

2013-11-07 Thread Robin St . Clair

Anssi

The last time I checked the use of IN, all the records from the database 
in the query were brought back to the workstation, rather than being 
processed on the backend and only the results returned to the workstation.


Have there been changes that carry out the entire query on the backend? 
What has changed to cause you to prefer the use of the IN statement?


R+C
On 08/11/2013 05:55, akaariai wrote:

On Sunday, November 3, 2013 1:48:07 PM UTC+2, Robin St.Clair wrote:

*IN*

  * if using Django avoid the IN operation at all costs


If there are potentially more than 15 items in the list, rework
the IN as a JOIN against whatever the source of the keys is


I don't necessarily agree with everything else said in the post, but 
this one is just plain wrong. It is completely OK to use 
__in=queryset. In fact, it is recommended in cases where the alternate 
is using join + distinct.


 - Anssi
--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-users+unsubscr...@googlegroups.com.

To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/6b5f9e3a-2e41-47b5-b0d6-94473fe323b8%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.


--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/BLU0-SMTP357C2324CC6BEC3F975EB4AE2F20%40phx.gbl.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query reduction

2013-11-07 Thread akaariai
On Sunday, November 3, 2013 1:48:07 PM UTC+2, Robin St.Clair wrote:
>
> *IN*
>
>- if using Django avoid the IN operation at all costs
>
>
> If there are potentially more than 15 items in the list, rework the IN as 
> a JOIN against whatever the source of the keys is
>

I don't necessarily agree with everything else said in the post, but this 
one is just plain wrong. It is completely OK to use __in=queryset. In fact, 
it is recommended in cases where the alternate is using join + distinct.

 - Anssi 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/6b5f9e3a-2e41-47b5-b0d6-94473fe323b8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query reduction

2013-11-04 Thread Apostolos Bessas
On Sat, Nov 2, 2013 at 4:50 PM, Daniele Procida  wrote:
>
> But, the real killer is the combination of ordering (in the queryset or on 
> the model, it doesn't matter) with the distinct() - as soon as one is removed 
> from the equation, the execution time drops to around 250ms.
>
> That's for 55000 BibliographicRecords created by that last operation (before 
> distinct() is applied; distinct() reduces them to 28000).


Do you happen to use PostgreSQL? This could be a case of a
"non-optimal" configuration, that makes PostgreSQL use the disk to do
the sorting. Take a look at
http://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-WORK-MEM.

Apostolis

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEa3b%2BoCFp3oPwtcR-uyoERW3YwHR4aXLubp7WUpa%2BHtYNhvmw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


RE: Complex query reduction

2013-11-03 Thread Robin St . Clair
Hi
You are running into typical production Django problems - works fine in 
testing, terrible in production.
Firstly
If you use attributed many-to-manys, create the table outside of Django and use 
raw SQL.Create a primary key on the two things being related (usually larger 
table first).
Multi-column indicesTwo choicesA single index that includes both (do this 
outside Django)Two indices, one for each columnThe size will be roughly 
equivalent between the two
Single composite indexsmaller than two indexes.faster to update.accelerates a 
more restricted range of queries.An index on (A, B) does nothing for queries 
just using BBut on the queries it does accelerate, it will be faster.
Two indiceslargerslower to updatewill help on queries on both columns, but not 
as much as a single composite indexwill help accelerate queries on either of 
the columnscan be expressed directly in Django
Helpful Toolpg_stat_activity shows a wealth of useful information - including
which indexes are actually being used
If an index is not being used (or not being used very often) drop itif you are 
surprised that it is not being used, find out why
Basic Performance Rules1. Do not iterate over QuerySets2. If you think you have 
to iterate over a QuerySet, see rule #13. If you are absolutely, certainly, 
100% positive that the only possible solution to your problem is iterating over 
a QuerySet, see rule #3
Iteration - A Health Warning• Ignores Django’s lazy-evaluation mechanism and 
copies everything into local memory• Copies data from the database just to 
process it locally• Does filtration or summation in the application that is 
processed more efficiently in the database.• Databases are good at this sort of 
stuff let the DB do it
Alternatives to Iteration• QuerySet.update()• cursor.execute(“UPDATE 
reader_hours ...”)• Stored procedures
Take in the code below
How much many objects are in memory at point A?
qs = Orders.objects.all() # There are about 2,500,000 rows in “orders”
for order in qs:order.age_in_days += 1 # POINT A order.save()
Answer  - 2,500,000
Why Is This?
Django does lazy evaluation… (everyone tells me so!)The Django code carefully 
asks for a slice of 100 objectswhich trickles down through lots of really 
convoluted Python to psycopg2which dutifully asks for 100 rows from 
Postgreswhich sends all 2,500,000 over the wire
Solution - Named CursorsThe protocol between the Postgres client and server 
only does partial sends when using named cursorspsycopg2 fully supports named 
cursorsDjango doesn’t use named cursorsTherefore, the first time you ask for any 
object in a QuerySet, you get all of themThis is a very good reason not to ask 
for large result sets
INif using Django avoid the IN operation at all costs
If there are potentially more than 15 items in the list, rework the IN as a 
JOIN against whatever the source of the keys is

At some stage you are going to have to get to grips with SQL, might as well be 
sooner than later
R+C
PS some time ago one of my clients was only able to process 4 transactons a 
second against their very complicated DB. They needed to process 70+ 
transactions per second. After 3 days of code analysis we had the transaction 
rate up to 240 per second and so could get rid of the 6 week backlog that had 
developed. Front end coders didn't consider the DBMS to be importantDate: Sun, 
3 Nov 2013 02:37:09 -0800
From: akaar...@gmail.com
To: django-users@googlegroups.com
Subject: Re: Complex query reduction

You should rewrite the query into a form that doesn't require distinct. In 
general, when you see a query that has joins and DISTINCT, that should be an 
alarm bell that something isn't written correctly in the query. Unfortunately 
Django's ORM generates such queries, and that isn't easy to fix as there are 
databases that like DISTINCT more than rewriting the query to use subqueries.

In any case, you should check how to write the query without the need of 
distinct. Something like this should work:

self.items = BibliographicRecord.objects.listable_objects().filter(

authored__researcher__in=researchers,

)

self.items = BibliographicRecord.objects.listable_objects().filter(

pk__in=self.items.values_list('pk')

)

But maybe you can push the __in to deeper into the authored__researches 
lookup...

 - Anssi

On Saturday, November 2, 2013 4:50:42 PM UTC+2, Daniele Procida wrote:On Fri, 
Nov 1, 2013, Javier Guerra Giraldez <jav...@guerrag.com> wrote:



>have you tried eliminating the second IN relationship?  something like

>

>entities = entity.get_descendants()

>

>items = BibliographicRecord.objects.filter

>(authored__researcher__person__member_of__entity__in=entities).distinct()



Indeed I have, but in that form it takes around 1770ms, compared to around 
1540ms in the original form. What I actually do is:



# breaking apart the queries allows the use of values_lists

entities = self.entity.get_descendants(

 

Re: Complex query reduction

2013-11-03 Thread akaariai
You should rewrite the query into a form that doesn't require distinct. In 
general, when you see a query that has joins and DISTINCT, that should be 
an alarm bell that something isn't written correctly in the query. 
Unfortunately Django's ORM generates such queries, and that isn't easy to 
fix as there are databases that like DISTINCT more than rewriting the query 
to use subqueries.

In any case, you should check how to write the query without the need of 
distinct. Something like this should work:

self.items = BibliographicRecord.objects.
listable_objects().filter( 
authored__researcher__in=researchers, 
)
self.items = BibliographicRecord.objects.
listable_objects().filter( 
pk__in=self.items.values_list('pk') 
)

But maybe you can push the __in to deeper into the authored__researches 
lookup...

 - Anssi

On Saturday, November 2, 2013 4:50:42 PM UTC+2, Daniele Procida wrote:
>
> On Fri, Nov 1, 2013, Javier Guerra Giraldez  
> wrote: 
>
> >have you tried eliminating the second IN relationship?  something like 
> > 
> >entities = entity.get_descendants() 
> > 
> >items = BibliographicRecord.objects.filter 
> >(authored__researcher__person__member_of__entity__in=entities).distinct() 
>
> Indeed I have, but in that form it takes around 1770ms, compared to around 
> 1540ms in the original form. What I actually do is: 
>
> # breaking apart the queries allows the use of values_lists 
> entities = self.entity.get_descendants( 
> include_self=True 
> ).values_list('id', flat=True) 
>
> # and the set() here is about 230ms faster than putting a distinct() 
> on 
> # the first query 
> researchers = set(Researcher.objects.filter( 
> person__entities__in=entities 
> ).values_list('person', flat=True)) 
>
> self.items = BibliographicRecord.objects.listable_objects().filter( 
> authored__researcher__in=researchers, 
> ).distinct() 
>
> I think that's partly because this way the SELECT doesn't have to grab all 
> the fields of publications_bibliographicrecord. 
>
> But, the real killer is the combination of ordering (in the queryset or on 
> the model, it doesn't matter) with the distinct() - as soon as one is 
> removed from the equation, the execution time drops to around 250ms. 
>
> That's for 55000 BibliographicRecords created by that last operation 
> (before distinct() is applied; distinct() reduces them to 28000). 
>
> That seems excessive to me. 
>
> BibliographicRecord has a custom primary key, and its id fields look like 
> "d9ce7e2f-663e-4fc6-8448-b214c6915aed:web-of-science". Could that be 
> implicated in performance? 
>
> Daniele 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/8208cc22-cc94-4fe6-9245-709bdc42647f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query reduction

2013-11-02 Thread Daniele Procida
On Fri, Nov 1, 2013, Javier Guerra Giraldez  wrote:

>have you tried eliminating the second IN relationship?  something like
>
>entities = entity.get_descendants()
>
>items = BibliographicRecord.objects.filter
>(authored__researcher__person__member_of__entity__in=entities).distinct()

Indeed I have, but in that form it takes around 1770ms, compared to around 
1540ms in the original form. What I actually do is:

# breaking apart the queries allows the use of values_lists
entities = self.entity.get_descendants(
include_self=True
).values_list('id', flat=True)

# and the set() here is about 230ms faster than putting a distinct() on 
# the first query
researchers = set(Researcher.objects.filter(
person__entities__in=entities
).values_list('person', flat=True))

self.items = BibliographicRecord.objects.listable_objects().filter(
authored__researcher__in=researchers,
).distinct()

I think that's partly because this way the SELECT doesn't have to grab all the 
fields of publications_bibliographicrecord.

But, the real killer is the combination of ordering (in the queryset or on the 
model, it doesn't matter) with the distinct() - as soon as one is removed from 
the equation, the execution time drops to around 250ms.

That's for 55000 BibliographicRecords created by that last operation (before 
distinct() is applied; distinct() reduces them to 28000).

That seems excessive to me. 

BibliographicRecord has a custom primary key, and its id fields look like 
"d9ce7e2f-663e-4fc6-8448-b214c6915aed:web-of-science". Could that be implicated 
in performance?

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/20131102145042.868061817%40smtp.modern-world.net.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query reduction

2013-11-01 Thread Javier Guerra Giraldez
On Fri, Nov 1, 2013 at 12:45 PM, Daniele Procida  wrote:
> In practice I use some tweaks (such as values_list) to speed this up, and 
> caching, but the fundamental pattern is the same. It's slow, because there 
> are 30 thousand BibliographicRecords.


the total number of records shouldn't matter.  more important is the
number of selected records at some points in the query.  i'd guess the
number of chosen entities (those that _are_ descendant of the first
one), and the number of chosen researches (those that _are_ members of
those entities) should be the most significant quantities.

the point is that most DB optimizers are somewhat shy to shuffle
conditions around IN operators.  but it might be easier to work on a
small (a few tens of values?) IN condition than on a very long JOIN
chain.

>
> I'm trying to rearrange the construction of the query in different ways to 
> find a speed improvement. I don't think that either prefetch_related or 
> select_related will help here, but perhaps there are some other tools that 
> would.


have you tried eliminating the second IN relationship?  something like

entities = entity.get_descendants()

items = 
BibliographicRecord.objects.filter(authored__researcher__person__member_of__entity__in=entities).distinct()

that should turn most of the query in a long sequence of "INNER JOIN",
giving the optimizer an easier job, avoiding the IN operation on
research records.  if the query chooses too many researches, this
could be a big part of the slowness.

-- 
Javier

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFkDaoSVg3K44wRUnOTrinqXZbVKm1ZZLHxFPB8LdgrzBtxViA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query

2013-09-09 Thread Drew Ferguson
On Mon, 9 Sep 2013 06:57:46 -0700 (PDT)
Yegor Roganov  wrote:

> Thanks for you replies.
> But I wonder how real production web sites deal with this problem. The 
> problem is that I want to display about 30 topics per page which would 
> result in 30+1 queries if implemented naively. If I write the required 
> query in raw SQL, it should be faster, but probably not very fast
> anyway. Also I don't know how to cache properly since topics should be
> ordered by most recent posts (that is, if someone posts to topic on
> third page, this topic should become the first one on the first page).
> 

If you were using Postgres or similar, you could create some triggers
and/or stored-procedures that maintained an active list of 30 target
items - perhaps with a flag in one of the database tables. Then create a
view to generate the list of 30 items you needed.

This way the ORM & Django SQL access is reduced to a single query on the
view. The heavy lifting is all done server-side as comments/articles are
saved

Could this work?
-- 
Drew Ferguson

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query

2013-09-09 Thread Yegor Roganov
Thanks for you replies.
But I wonder how real production web sites deal with this problem. The 
problem is that I want to display about 30 topics per page which would 
result in 30+1 queries if implemented naively. If I write the required 
query in raw SQL, it should be faster, but probably not very fast anyway. 
Also I don't know how to cache properly since topics should be ordered by 
most recent posts (that is, if someone posts to topic on third page, this 
topic should become the first one on the first page).

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query

2013-09-09 Thread akaariai
On Monday, September 9, 2013 12:13:09 AM UTC+3, Yegor Roganov wrote:
>
> I found out this is a so called "top n per group" problem and it's not 
> that easy to implement in django. Maybe someone could help me to find 
> another way (for example, alter the schema in some way)?
>

This is something that would be nice to support with prefetch_related(). 
This is exactly the type of query where ORM abstraction would be very 
welcome. Writing this in SQL isn't easy, and the most efficient way to 
write the query varies a lot depending on the used database backend.

Unfortunately I don't see other solutions than running multiple queries or 
using raw SQL.

 - Anssi


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query

2013-09-08 Thread Jani Tiainen
Problem is that you have to be able to express it in SQL. And thus ther is not 
much you can do in SQL to get what you actually wanted.

Considering amount of the data fetched - it's relatively low query count and 
unless you're hitting very busy site you wont notice much of difference doing 
this in 6 queries.

One to fetch last 5 topics and then 5 to get that 20 latest posts per topic.

When you later may hit really busy site you can cache list for a short time to 
relieve load on db.

On Sun, 8 Sep 2013 14:13:09 -0700 (PDT)
Yegor Roganov  wrote:

> I found out this is a so called "top n per group" problem and it's not that 
> easy to implement in django. Maybe someone could help me to find another 
> way (for example, alter the schema in some way)?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query

2013-09-08 Thread Yegor Roganov
I found out this is a so called "top n per group" problem and it's not that 
easy to implement in django. Maybe someone could help me to find another 
way (for example, alter the schema in some way)?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex query

2013-09-08 Thread Nick Apostolakis

On 08/09/2013 08:17 μμ, Yegor Roganov wrote:

I'm developing a forum and my db schema  is roughly equivalent to the
following:
class Topic(models.Model):
 head = models.CharField()
 body = models.TextField()
 created = models.DateTimeField(auto_now_add=True)

class Post(models.Model):
 body = models.TextField()
 topic = models.ForeignKey(Topic)
 created = models.DateTimeField(auto_now_add=True)

The question is: how to construct a query to fetch 20 latest Topics with 5
latest Posts for each of these Topics ? As I understood, prefetch_related
isn't suitable since it cannot limit the number of related objects.


It looks like you are looking for a subquery

maybe this will be useful

http://stackoverflow.com/questions/8217490/query-of-a-subquery-in-django

--
 --
 Nick Apostolakis
 Msc in IT, University of Glasgow
 e-mail: nicka...@oncrete.gr
 Web Site: http://nick.oncrete.gr
 --

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Complex Query help - get items from the org to which the user belongs

2011-07-07 Thread Venkatraman S
On Thu, Jul 7, 2011 at 3:49 PM, DrBloodmoney  wrote:

> On Thu, Jul 7, 2011 at 12:04 AM, Venkatraman S  wrote:
> > I am doing some bechmarking on the performance of both the versions of
> the
> > query:
> > 1.
> >
> Item.objects.filter(created_by__employees__org__in=u.employees_set.all().values_list('org'))
> > and
> > 2. Items.objects.extra(where=['created_by_id in (select e.user_id from
> > myapp_employees e, myapp_organization o where e.org_id = o.id and o.id =
> > (select o2.id from myapp_organization o2, myapp_employees e2 where
> e2.org_id
> > = o2.id and e2.user_id=3 and e2.deleted=False)) '])
> >
> > I am seeing #1 to be faster(when i view from DDT). Will update in the
> next
> > few days or probably early next week.
> >
> > -V
>
> Personally, I'd add a FK on the Item to the Employee, particularly if
> they're in the same app (Actually I'd replace the FK to User with one
> to Employee). I suspect that is a simplified models.py so I don't
> actually know if there would be a requirement to FK to User. (I also
> try to limit my code touching django.contrib.auth since using it is my
> biggest pain point for django).
>
>
Cant actually, as the Item can be created by both employees, and
'outsiders'.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-07 Thread DrBloodmoney
On Thu, Jul 7, 2011 at 12:04 AM, Venkatraman S  wrote:
> I am doing some bechmarking on the performance of both the versions of the
> query:
> 1.
> Item.objects.filter(created_by__employees__org__in=u.employees_set.all().values_list('org'))
> and
> 2. Items.objects.extra(where=['created_by_id in (select e.user_id from
> myapp_employees e, myapp_organization o where e.org_id = o.id and o.id =
> (select o2.id from myapp_organization o2, myapp_employees e2 where e2.org_id
> = o2.id and e2.user_id=3 and e2.deleted=False)) '])
>
> I am seeing #1 to be faster(when i view from DDT). Will update in the next
> few days or probably early next week.
>
> -V

Personally, I'd add a FK on the Item to the Employee, particularly if
they're in the same app (Actually I'd replace the FK to User with one
to Employee). I suspect that is a simplified models.py so I don't
actually know if there would be a requirement to FK to User. (I also
try to limit my code touching django.contrib.auth since using it is my
biggest pain point for django).

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread Venkatraman S
I am doing some bechmarking on the performance of both the versions of the
query:
1.
Item.objects.filter(created_by__employees__org__in=u.employees_set.all().values_list('org'))
and
2. Items.objects.extra(where=['created_by_id in (select e.user_id from
myapp_employees e, myapp_organization o where e.org_id = o.id and o.id =
(select o2.id from myapp_organization o2, myapp_employees e2 where e2.org_id
= o2.id and e2.user_id=3 and e2.deleted=False)) '])

I am seeing #1 to be faster(when i view from DDT). Will update in the next
few days or probably early next week.

-V

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread Venkatraman S
On Wed, Jul 6, 2011 at 8:09 PM, Marc Aymerich  wrote:

> I swear this time it will work!
>
> Item.objects.filter(created_by__employees__org__in=u.employees_set.all().values_list('org'))
>
>
I cried on seeing this ;) Thanks a tonne.  Guess this will go into the
django hall of fame!
But i have a Q : the query that i pinged (using 'where') works, so how
efficient is your version? (am parallely checking this using DDT).

-V

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread Marc Aymerich
On Wed, Jul 6, 2011 at 2:46 PM, Venkatraman S  wrote:
>
>
> On Wed, Jul 6, 2011 at 2:10 PM, Marc Aymerich  wrote:
>>
>> This one should work:
>> Items.objects.filters(created_by__employee__org=A.org)
>
> Nope. Emp has a FK for User, not the other way round.

I swear this time it will work!
Item.objects.filter(created_by__employees__org__in=u.employees_set.all().values_list('org'))
--
Marc

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread Tom Evans
On Wed, Jul 6, 2011 at 2:40 PM, akaariai  wrote:
> Maybe this?
>
> user = current_user
> user_org_employees = Employees.objects.filter(org=user.org)
> user_org_items =
> Items.objects.filter(created_by__in=user_org_employees)
>

No, users can belong to multiple organizations, so no using user.org.
Something like this:

orgs = Organization.objects.filter(employees__user=user)
users = User.objects.filter(employee__org__in=orgs)
items = Item.objects.filter(created_by__in=users)

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread akaariai
On Jul 5, 10:06 pm, Venkatraman S  wrote:
> I tried asking around in IRC, and stumbled on a few possible solutions,
> would be great if someone shed some more light:
>
> I have the following models:
>
> class Organization(models.Model):
>   name                = models.CharField(max_length=100, blank=False)
>
> class Employees(models.Model):
>   org                 = models.ForeignKey(Organization,
> related_name='employees')
>   user                = models.ForeignKey(User)
>   name                = models.CharField(max_length=100, blank=False)
>
> class Item(models.Model):
>   name                = models.CharField(max_length=100, blank=False)
>   created_by          =
> models.ForeignKey(User,related_name='created_by_whom')
>
> Problem : I need to get all Items from the Organization to which current
> User belongs to.
> So basically, get all employees from the Org that the current User belongs
> to, and then get all items created by these employees. (So, this query
> should also include the current User).

Maybe this?

user = current_user
user_org_employees = Employees.objects.filter(org=user.org)
user_org_items =
Items.objects.filter(created_by__in=user_org_employees)

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread Venkatraman S
On Wed, Jul 6, 2011 at 2:10 PM, Marc Aymerich  wrote:

> This one should work:
> Items.objects.filters(created_by__employee__org=A.org)
>

Nope. Emp has a FK for User, not the other way round.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-06 Thread Marc Aymerich
On Wed, Jul 6, 2011 at 4:46 AM, Venkatraman S  wrote:

>
> On Wed, Jul 6, 2011 at 2:26 AM, Marc Aymerich  wrote:
>
>> Say there are 3 Users in an org : A, B and C with each creating 3,4,5
>> items respectively, and 'A' is the current user; then i need a query which
>> returns all these items(i.e, 12 items) when i supply A..
>>
>> Item.objects.filter(employees__org=A.org)
>> This works¿?
>>
>
> I dont think this would work; Items doesnt have an FK for Emps.
>

Sorry, I answered too fast.

This one should work:
Items.objects.filters(created_by__employee__org=A.org)


-- 
Marc

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-05 Thread Venkatraman S
On Wed, Jul 6, 2011 at 8:39 AM, Venkatraman S  wrote:

>
> On Wed, Jul 6, 2011 at 12:36 AM, Venkatraman S  wrote:
>
>> I tried asking around in IRC, and stumbled on a few possible solutions,
>> would be great if someone shed some more light:
>>
>> I have the following models:
>>
>> class Organization(models.Model):
>>   name= models.CharField(max_length=100, blank=False)
>>
>> class Employees(models.Model):
>>   org = models.ForeignKey(Organization,
>> related_name='employees')
>>   user= models.ForeignKey(User)
>>   name= models.CharField(max_length=100, blank=False)
>>
>> class Item(models.Model):
>>   name= models.CharField(max_length=100, blank=False)
>>   created_by  =
>> models.ForeignKey(User,related_name='created_by_whom')
>>
>> Problem : I need to get all Items from the Organization to which current
>> User belongs to.
>> So basically, get all employees from the Org that the current User belongs
>> to, and then get all items created by these employees. (So, this query
>> should also include the current User).
>>
>> Say there are 3 Users in an org : A, B and C with each creating 3,4,5
>> items respectively, and 'A' is the current user; then i need a query which
>> returns all these items(i.e, 12 items) when i supply A..
>>
>
>
> This is the equivalent raw sql:
> select count(1)
> from myapp_items a
> where a.created_by_id in
> (
> select e.user_id
> from myapp_employees e, myapp_organization o
> where e.org_id = o.id
> and o.id = (select o2.id from myapp_organization o2, myapp_employees e2
> where e2.org_id = o2.id and e2.user_id=<>)
> )
>
>
And this would be a probable query:
Items.objects.extra(where=['created_by_id in (select e.user_id from
myapp_employees e, myapp_organization o where e.org_id = o.id and o.id =
(select o2.id from myapp_organization o2, myapp_employees e2 where e2.org_id
= o2.id and e2.user_id=3)) ']).count()

Can some expert in django-ORM comment on this? (can the same be done without
a 'where' clause)

As  someone pointed out in IRC, i want to go up, down and then sideways in
this query ;)

-V

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-05 Thread Venkatraman S
On Wed, Jul 6, 2011 at 12:36 AM, Venkatraman S  wrote:

> I tried asking around in IRC, and stumbled on a few possible solutions,
> would be great if someone shed some more light:
>
> I have the following models:
>
> class Organization(models.Model):
>   name= models.CharField(max_length=100, blank=False)
>
> class Employees(models.Model):
>   org = models.ForeignKey(Organization,
> related_name='employees')
>   user= models.ForeignKey(User)
>   name= models.CharField(max_length=100, blank=False)
>
> class Item(models.Model):
>   name= models.CharField(max_length=100, blank=False)
>   created_by  =
> models.ForeignKey(User,related_name='created_by_whom')
>
> Problem : I need to get all Items from the Organization to which current
> User belongs to.
> So basically, get all employees from the Org that the current User belongs
> to, and then get all items created by these employees. (So, this query
> should also include the current User).
>
> Say there are 3 Users in an org : A, B and C with each creating 3,4,5 items
> respectively, and 'A' is the current user; then i need a query which returns
> all these items(i.e, 12 items) when i supply A..
>


This is the equivalent raw sql:
select count(1)
from myapp_items a
where a.created_by_id in
(
select e.user_id
from myapp_employees e, myapp_organization o
where e.org_id = o.id
and o.id = (select o2.id from myapp_organization o2, myapp_employees e2
where e2.org_id = o2.id and e2.user_id=<>)
)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-05 Thread Venkatraman S
On Wed, Jul 6, 2011 at 2:26 AM, Marc Aymerich  wrote:

> Say there are 3 Users in an org : A, B and C with each creating 3,4,5 items
> respectively, and 'A' is the current user; then i need a query which returns
> all these items(i.e, 12 items) when i supply A..
>
> Item.objects.filter(employees__org=A.org)
> This works¿?
>

I dont think this would work; Items doesnt have an FK for Emps.

-V

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query help - get items from the org to which the user belongs

2011-07-05 Thread Marc Aymerich
On Tue, Jul 5, 2011 at 9:06 PM, Venkatraman S  wrote:

> I tried asking around in IRC, and stumbled on a few possible solutions,
> would be great if someone shed some more light:
>
> I have the following models:
>
> class Organization(models.Model):
>   name= models.CharField(max_length=100, blank=False)
>
> class Employees(models.Model):
>   org = models.ForeignKey(Organization,
> related_name='employees')
>   user= models.ForeignKey(User)
>   name= models.CharField(max_length=100, blank=False)
>
> class Item(models.Model):
>   name= models.CharField(max_length=100, blank=False)
>   created_by  =
> models.ForeignKey(User,related_name='created_by_whom')
>
> Problem : I need to get all Items from the Organization to which current
> User belongs to.
> So basically, get all employees from the Org that the current User belongs
> to, and then get all items created by these employees. (So, this query
> should also include the current User).
>
> Say there are 3 Users in an org : A, B and C with each creating 3,4,5 items
> respectively, and 'A' is the current user; then i need a query which returns
> all these items(i.e, 12 items) when i supply A..
>

Item.objects.filter(employees__org=A.org)
This works¿?


-- 
Marc

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: complex query

2011-07-04 Thread Andre Terra
Hello

First, I must recommend that you use django-taggit, simpletag or any
other tagging app, otherwise you'll just be reinventing the wheel.

As for your question, you should read the aggregation docs. I don't
have the link handy, but this should be correct:
http://django.me/aggregation

Sincerely,
Andre Terra

On 7/3/11, Jonas Geiregat  wrote:
> Hello,
>
> I have the following models
>
> class Book(models.Model):
>   author = models.ManyToManyField(Author)
>   pub_date = models.DateTimeField()
>   ...
>
> class Author(models.Model):
>   tag = models.ManyToManyField(Tag)
>
> class Tag(models.Model):
>   name = models.CharField(max_length=20)
>
> What I want to query for is:
>
> The most used Tags (and their count) for all books who's pub_date is greater
> then today.
>
> I can solve the last part
> Book.objects.filter(pub_date__gt=datetime.date.today())
>
> But how do I count all the tags for all these books ?
>
> Jonas Geiregat
> jo...@geiregat.org
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>

-- 
Sent from my mobile device

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread gontran
Now that I changed my model, the query is a little bit different.
So, my models:

UserProfile(models.Model):
user = models.OneToOneField(User)
company = models.ForeignKey(Company)

Company(models.Model):
...some fields...

Product(models.Model):
...some fields...

ProductLicence(models.Model):
product = models.ForeignKey(Product)
company = models.ForeignKey(Company)

News(models.Model):
   related_products = models.ManyToManyField(Product)


And my query:
news =
News.objects.filter(related_products__productlicence__in=user.get_profile().company.productlicence_set.all()).distinct()


I hope that it will be useful to somebody.

Regards,

Gontran

On 29 mar, 10:34, Kenneth Gonsalves  wrote:
> On Tue, 2011-03-29 at 01:17 -0700, bruno desthuilliers wrote:
> > On 29 mar, 09:33, Kenneth Gonsalves  wrote:
> > > Product is not linked to them - they are linked to Product.
>
> > Sorry but I dont see the point here - you can follow a relationship
> > both way.
>
> pointless nitpick - apologies
> --
> regards
> KGhttp://lawgon.livejournal.com
> Coimbatore LUG roxhttp://ilugcbe.techstud.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread Kenneth Gonsalves
On Tue, 2011-03-29 at 01:17 -0700, bruno desthuilliers wrote:
> On 29 mar, 09:33, Kenneth Gonsalves  wrote:
> > Product is not linked to them - they are linked to Product.
> 
> Sorry but I dont see the point here - you can follow a relationship
> both way. 

pointless nitpick - apologies
-- 
regards
KG
http://lawgon.livejournal.com
Coimbatore LUG rox
http://ilugcbe.techstud.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread gontran
Hi Bruno,

yes you're right. I should change my model in the way you mentioned
it. I saw that a couple of minutes ago.

And to answer to my initial question, I might have found the answer.
The query would be:

news = News.objects.filter(related_products__productlicence__in =
user.get_profile().company.product_licences.all()).distinct()

On 29 mar, 10:24, bruno desthuilliers 
wrote:
> On 29 mar, 09:11, gontran  wrote:
>
> > Hi everybody,
>
> > considering the folowing models:
>
> > UserProfile(models.Model):
> >     user = models.OneToOneField(User)
> >     company = models.ForeignKey(Company)
>
> > Company(models.Model):
> >     product_licences = models.manyToManyField(ProductLicence)
>
> Sorry if I missed something, but are you sure you want a m2m
> relationship here ??? This means that a same licence can "belong" to
> many companies, which seems rather weird to me. As far as I'm
> concerned I'd make "company" a foreign key in ProductLicence - or I
> just don't understand your definition of what "licence" is ???
>
> > Product(models.Model):
> >     ...some fields...
>
> > ProductLicence(models.Model):
> >     product = models.ForeignKey(Product)
>
> (snip)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread bruno desthuilliers


On 29 mar, 09:11, gontran  wrote:
> Hi everybody,
>
> considering the folowing models:
>
> UserProfile(models.Model):
>     user = models.OneToOneField(User)
>     company = models.ForeignKey(Company)
>
> Company(models.Model):
>     product_licences = models.manyToManyField(ProductLicence)


Sorry if I missed something, but are you sure you want a m2m
relationship here ??? This means that a same licence can "belong" to
many companies, which seems rather weird to me. As far as I'm
concerned I'd make "company" a foreign key in ProductLicence - or I
just don't understand your definition of what "licence" is ???


> Product(models.Model):
>     ...some fields...
>
> ProductLicence(models.Model):
>     product = models.ForeignKey(Product)
>


(snip)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread bruno desthuilliers
On 29 mar, 09:33, Kenneth Gonsalves  wrote:
> Product is not linked to them - they are linked to Product.

Sorry but I dont see the point here - you can follow a relationship
both way.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread gontran
Ok. I 'll try again
Thank you very much for your consideration.

Regards,

Gontran

On 29 mar, 09:33, Kenneth Gonsalves  wrote:
> Product is not linked to them - they are linked to Product. I do not
> think you can do it in one query because there are ManyToMany fields
> which have to be retrieved and then you have to iterate over them.
> Certainly more than one query.
>
>
>
>
>
>
>
>
>
> On Tue, 2011-03-29 at 00:24 -0700, gontran wrote:
> > Yes: ProductLicence and News
>
> > On 29 mar, 09:19, Kenneth Gonsalves  wrote:
> > > On Tue, 2011-03-29 at 00:11 -0700, gontran wrote:
> > > > UserProfile(models.Model):
> > > >     user = models.OneToOneField(User)
> > > >     company = models.ForeignKey(Company)
>
> > > > Company(models.Model):
> > > >     product_licences = models.manyToManyField(ProductLicence)
>
> > > > Product(models.Model):
> > > >     ...some fields...
>
> > > > ProductLicence(models.Model):
> > > >     product = models.ForeignKey(Product)
>
> > > > News(models.Model):
> > > >    related_products = models.ManyToManyField(Product)
>
> > > > I already know how to retrieve all distinct products for which the
> > > > company of the user owns licences, but now, what I want to do is to
> > > > retrieve all news, for a given user, that are related to products, for
> > > > which the company of the user owns licences.
>
> > > Product is not linked to any model?
> > > --
> > > regards
> > > KGhttp://lawgon.livejournal.com
> > > Coimbatore LUG roxhttp://ilugcbe.techstud.org/
>
> --
> regards
> KGhttp://lawgon.livejournal.com
> Coimbatore LUG roxhttp://ilugcbe.techstud.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread Kenneth Gonsalves
Product is not linked to them - they are linked to Product. I do not
think you can do it in one query because there are ManyToMany fields
which have to be retrieved and then you have to iterate over them.
Certainly more than one query.

On Tue, 2011-03-29 at 00:24 -0700, gontran wrote:
> Yes: ProductLicence and News
> 
> On 29 mar, 09:19, Kenneth Gonsalves  wrote:
> > On Tue, 2011-03-29 at 00:11 -0700, gontran wrote:
> > > UserProfile(models.Model):
> > > user = models.OneToOneField(User)
> > > company = models.ForeignKey(Company)
> >
> > > Company(models.Model):
> > > product_licences = models.manyToManyField(ProductLicence)
> >
> > > Product(models.Model):
> > > ...some fields...
> >
> > > ProductLicence(models.Model):
> > > product = models.ForeignKey(Product)
> >
> > > News(models.Model):
> > >related_products = models.ManyToManyField(Product)
> >
> > > I already know how to retrieve all distinct products for which the
> > > company of the user owns licences, but now, what I want to do is to
> > > retrieve all news, for a given user, that are related to products, for
> > > which the company of the user owns licences.
> >
> > Product is not linked to any model?
> > --
> > regards
> > KGhttp://lawgon.livejournal.com
> > Coimbatore LUG roxhttp://ilugcbe.techstud.org/
> 


-- 
regards
KG
http://lawgon.livejournal.com
Coimbatore LUG rox
http://ilugcbe.techstud.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread gontran
Yes: ProductLicence and News

On 29 mar, 09:19, Kenneth Gonsalves  wrote:
> On Tue, 2011-03-29 at 00:11 -0700, gontran wrote:
> > UserProfile(models.Model):
> >     user = models.OneToOneField(User)
> >     company = models.ForeignKey(Company)
>
> > Company(models.Model):
> >     product_licences = models.manyToManyField(ProductLicence)
>
> > Product(models.Model):
> >     ...some fields...
>
> > ProductLicence(models.Model):
> >     product = models.ForeignKey(Product)
>
> > News(models.Model):
> >    related_products = models.ManyToManyField(Product)
>
> > I already know how to retrieve all distinct products for which the
> > company of the user owns licences, but now, what I want to do is to
> > retrieve all news, for a given user, that are related to products, for
> > which the company of the user owns licences.
>
> Product is not linked to any model?
> --
> regards
> KGhttp://lawgon.livejournal.com
> Coimbatore LUG roxhttp://ilugcbe.techstud.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex query

2011-03-29 Thread Kenneth Gonsalves
On Tue, 2011-03-29 at 00:11 -0700, gontran wrote:
> UserProfile(models.Model):
> user = models.OneToOneField(User)
> company = models.ForeignKey(Company)
> 
> Company(models.Model):
> product_licences = models.manyToManyField(ProductLicence)
> 
> Product(models.Model):
> ...some fields...
> 
> ProductLicence(models.Model):
> product = models.ForeignKey(Product)
> 
> News(models.Model):
>related_products = models.ManyToManyField(Product)
> 
> I already know how to retrieve all distinct products for which the
> company of the user owns licences, but now, what I want to do is to
> retrieve all news, for a given user, that are related to products, for
> which the company of the user owns licences.
> 
> 

Product is not linked to any model?
-- 
regards
KG
http://lawgon.livejournal.com
Coimbatore LUG rox
http://ilugcbe.techstud.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: Complex Query Question

2009-11-20 Thread Javier Guerra
mystand wrote:
> I need to do a query that selects all Events that a logged in user can edit.
> They can edit an event if they are in the default group, if they are in any
> of the other groups, or they are one of the users added to the event. 

first of all, it might be easier if you follow the database convention of 
naming your tables in singular.  in this case, your events model should be 
called 'Event' and not 'Events'.  that makes it more readable when you use 
singular in ForeignKey fields and plural in ManyToManyField's

in your case, the general strategy is to use Q() objects.  the idea is to 
define each possibility separately and then join them all with the '|' 
operator.  something like this (untested):

@login_required
def editable_groups(request):
user = request.user
evs_by_defgroup = Q(default_group__in=user.group_set)
evs_by_groups = Q(group__in=user.group_set)
evs_by_users = user.events_set

myevents = Events.objects.filter (evs_by_defgroup | evs_by_groups | 
evs_by_users)
return render_to_response ("template.html", {'myevents':myevents})


(i'm not sure about '|'ing the Q() objects with the queryset 'user.events_set', 
but i don't know how to express this as a Q() object)


-- 
Javier

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=.




Re: Complex query in model manager

2009-08-23 Thread CrabbyPete

or another words can I use a Q object in a model manager?

On Aug 23, 12:05 am, CrabbyPete  wrote:
> I have the following code in model manager
>
> class SpotMessageManager(models.Manager):
>
>       def for_user(self, user, start = 0, end = None, friends ):
>             messages = self.filter(user = user).order_by
> ('date_added').reverse()
>             ...
>
> friends is a list of users, and I want the query set to include the
> user and all the friends in the list. What is the best way to do this?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---



Re: complex query in limit_choices_to

2007-11-24 Thread leotr

On database level that should be done by UNIQUE constraints.
And your problem should be made on form level.
read about ChoiceField in newforms documentation
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---