Re: ORM from the command line

[Tim Johnson]

Thank you too Adam.

Great stuff!

On 4/9/20 7:43 AM, Adam Mičuda wrote:

Hi,
you can also write simple python script and run it in Django context 
via 
https://django-extensions.readthedocs.io/en/latest/runscript.html from 
`django-extensions` package.


Best regards.
Adam

čt 9. 4. 2020 v 17:14 odesílatel Tim Johnson > napsal:


Thank you Andréas. I have come across that too, after my OT.

This is definitely what I was looking for.

cheers

On 4/9/20 2:05 AM, Andréas Kühne wrote:

Hi Tim,

What you probably should do is use a custom command on the
manage.py command interface. You till then get access to all of
djangos goodness - and it can be run from the command line.

See here:
https://docs.djangoproject.com/en/3.0/howto/custom-management-commands/

This is how I would handle it.

Regards,

Andréas


Den tors 9 apr. 2020 kl 00:18 skrev Tim Johnson
mailto:t...@akwebsoft.com>>:

using django.VERSION (2, 1, 5, 'final', 0) with

python 3.7.2 on ubuntu 16.04

I have a need for a "Housekeeping" application. It's usage
would be to

1) connect to a database, either mysql, mariadb or postgres

2) truncate two tables and repopulate them based on an
arbitrary data
structure such as a compound list or tuple.

Such an application would not need be and most preferably
should not be
part of a deployed website.

This should not be a very complicated endeavor. The simplest
method
might be to manually establish an ORM connection using
settings.py to
import the connection credentials. I am wondering if this is
possible.

However, I am unable to find documentation that would edify
me on
manually coding an ORM connection and clearing a database
without the
loading of django resources.

If such an approach is feasible, I would welcome URLs to
appropriate
documentation and/or discussion.

Using a model-view approach would be the simplest method, I
would think,
but there would be no need to have such a view deployed.
There is
probably a solution that would necessitate installing a
custom package
to be used from manage.py, such as
https://github.com/KhaledElAnsari/django-truncate and that
might be
complicated.

Comments are welcome

thanks

-- 
Tim

tj49.com 

-- 
You received this message because you are subscribed to the

Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
django-users+unsubscr...@googlegroups.com
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/4be7c6fc-d99e-7419-24ef-d54cbdf384d4%40akwebsoft.com.

-- 
You received this message because you are subscribed to the

Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-users+unsubscr...@googlegroups.com
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/CAK4qSCdSf%2BJ9ZufHavQqv-9ftOkOUwNODsFd-w6QQmzzK61j5Q%40mail.gmail.com

.


-- 
Tim

tj49.com  

-- 
You received this message because you are subscribed to the Google

Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-users+unsubscr...@googlegroups.com
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/35392bf4-d25a-95d6-fb32-bf09fd01e997%40akwebsoft.com

.

--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAB9%3DGXbjT24BoEtw-_mUc7wS8WtX9kSFQnfKTmCmWEfv5nTomQ%40mail.gmail.com 
.


--
Tim
tj49.com

--
You

Re: ORM from the command line

[Adam Mičuda]

Hi,
you can also write simple python script and run it in Django context via
https://django-extensions.readthedocs.io/en/latest/runscript.html from
`django-extensions` package.

Best regards.
Adam

čt 9. 4. 2020 v 17:14 odesílatel Tim Johnson  napsal:

> Thank you Andréas. I have come across that too, after my OT.
>
> This is definitely what I was looking for.
>
> cheers
> On 4/9/20 2:05 AM, Andréas Kühne wrote:
>
> Hi Tim,
>
> What you probably should do is use a custom command on the manage.py
> command interface. You till then get access to all of djangos goodness -
> and it can be run from the command line.
>
> See here:
> https://docs.djangoproject.com/en/3.0/howto/custom-management-commands/
>
> This is how I would handle it.
>
> Regards,
>
> Andréas
>
>
> Den tors 9 apr. 2020 kl 00:18 skrev Tim Johnson :
>
>> using django.VERSION (2, 1, 5, 'final', 0) with
>>
>> python 3.7.2 on ubuntu 16.04
>>
>> I have a need for a "Housekeeping" application. It's usage would be to
>>
>> 1) connect to a database, either mysql, mariadb or postgres
>>
>> 2) truncate two tables and repopulate them based on an arbitrary data
>> structure such as a compound list or tuple.
>>
>> Such an application would not need be and most preferably should not be
>> part of a deployed website.
>>
>> This should not be a very complicated endeavor. The simplest method
>> might be to manually establish an ORM connection using settings.py to
>> import the connection credentials. I am wondering if this is possible.
>>
>> However, I am unable to find documentation that would edify me on
>> manually coding an ORM connection and clearing a database without the
>> loading of django resources.
>>
>> If such an approach is feasible, I would welcome URLs to appropriate
>> documentation and/or discussion.
>>
>> Using a model-view approach would be the simplest method, I would think,
>> but there would be no need to have such a view deployed. There is
>> probably a solution that would necessitate installing a custom package
>> to be used from manage.py, such as
>> https://github.com/KhaledElAnsari/django-truncate and that might be
>> complicated.
>>
>> Comments are welcome
>>
>> thanks
>>
>> --
>> Tim
>> tj49.com
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/4be7c6fc-d99e-7419-24ef-d54cbdf384d4%40akwebsoft.com
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAK4qSCdSf%2BJ9ZufHavQqv-9ftOkOUwNODsFd-w6QQmzzK61j5Q%40mail.gmail.com
> 
> .
>
> --
> Timtj49.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/35392bf4-d25a-95d6-fb32-bf09fd01e997%40akwebsoft.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAB9%3DGXbjT24BoEtw-_mUc7wS8WtX9kSFQnfKTmCmWEfv5nTomQ%40mail.gmail.com.

Re: ORM from the command line

[Tim Johnson]

Thank you Andréas. I have come across that too, after my OT.

This is definitely what I was looking for.

cheers

On 4/9/20 2:05 AM, Andréas Kühne wrote:

Hi Tim,

What you probably should do is use a custom command on the manage.py 
command interface. You till then get access to all of djangos goodness 
- and it can be run from the command line.


See here:
https://docs.djangoproject.com/en/3.0/howto/custom-management-commands/

This is how I would handle it.

Regards,

Andréas


Den tors 9 apr. 2020 kl 00:18 skrev Tim Johnson >:


using django.VERSION (2, 1, 5, 'final', 0) with

python 3.7.2 on ubuntu 16.04

I have a need for a "Housekeeping" application. It's usage would be to

1) connect to a database, either mysql, mariadb or postgres

2) truncate two tables and repopulate them based on an arbitrary data
structure such as a compound list or tuple.

Such an application would not need be and most preferably should
not be
part of a deployed website.

This should not be a very complicated endeavor. The simplest method
might be to manually establish an ORM connection using settings.py to
import the connection credentials. I am wondering if this is possible.

However, I am unable to find documentation that would edify me on
manually coding an ORM connection and clearing a database without the
loading of django resources.

If such an approach is feasible, I would welcome URLs to appropriate
documentation and/or discussion.

Using a model-view approach would be the simplest method, I would
think,
but there would be no need to have such a view deployed. There is
probably a solution that would necessitate installing a custom
package
to be used from manage.py, such as
https://github.com/KhaledElAnsari/django-truncate and that might be
complicated.

Comments are welcome

thanks

-- 
Tim

tj49.com 

-- 
You received this message because you are subscribed to the Google

Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-users+unsubscr...@googlegroups.com
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/4be7c6fc-d99e-7419-24ef-d54cbdf384d4%40akwebsoft.com.

--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAK4qSCdSf%2BJ9ZufHavQqv-9ftOkOUwNODsFd-w6QQmzzK61j5Q%40mail.gmail.com 
.


--
Tim
tj49.com

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/35392bf4-d25a-95d6-fb32-bf09fd01e997%40akwebsoft.com.

Re: ORM from the command line

[Andréas Kühne]

Hi Tim,

What you probably should do is use a custom command on the manage.py
command interface. You till then get access to all of djangos goodness -
and it can be run from the command line.

See here:
https://docs.djangoproject.com/en/3.0/howto/custom-management-commands/

This is how I would handle it.

Regards,

Andréas


Den tors 9 apr. 2020 kl 00:18 skrev Tim Johnson :

> using django.VERSION (2, 1, 5, 'final', 0) with
>
> python 3.7.2 on ubuntu 16.04
>
> I have a need for a "Housekeeping" application. It's usage would be to
>
> 1) connect to a database, either mysql, mariadb or postgres
>
> 2) truncate two tables and repopulate them based on an arbitrary data
> structure such as a compound list or tuple.
>
> Such an application would not need be and most preferably should not be
> part of a deployed website.
>
> This should not be a very complicated endeavor. The simplest method
> might be to manually establish an ORM connection using settings.py to
> import the connection credentials. I am wondering if this is possible.
>
> However, I am unable to find documentation that would edify me on
> manually coding an ORM connection and clearing a database without the
> loading of django resources.
>
> If such an approach is feasible, I would welcome URLs to appropriate
> documentation and/or discussion.
>
> Using a model-view approach would be the simplest method, I would think,
> but there would be no need to have such a view deployed. There is
> probably a solution that would necessitate installing a custom package
> to be used from manage.py, such as
> https://github.com/KhaledElAnsari/django-truncate and that might be
> complicated.
>
> Comments are welcome
>
> thanks
>
> --
> Tim
> tj49.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/4be7c6fc-d99e-7419-24ef-d54cbdf384d4%40akwebsoft.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAK4qSCdSf%2BJ9ZufHavQqv-9ftOkOUwNODsFd-w6QQmzzK61j5Q%40mail.gmail.com.

Re: orm relation

[Dvs Khamele]

Hi We are worlds first affordable Python, Django Based Commpany,
And can help you in this and related tasks / projects immidiately mere at
$7 per hour
Best Regards,
Divyesh Khamele,
Pythonmate

On Sun, 1 Dec 2019 at 19:39, bill dexter <55dexte...@gmail.com> wrote:

> ELECT a.matr
>   ,[nom]
>   ,[prn]
>   ,cast([dat_nais] as date) as dat_nais
>   , (YEAR(getdate()) - YEAR(dat_nais)) as age
>   ,cast([dat_deces] as date) as dat_deces
>   ,cast([dat_imm] as date) as dat_imm
>   ,cast([dat_aj] as date) as dat_j
>   ,[cod_position]
>   ,YEAR(dat_imm) as anne_imm
>   ,YEAR(dat_encais) as ann_regl
>   ,[annee]
>   ,[cod_nat]
>   ,cast([dat_e
>
> Snc] as date) as dat_regl
>   ,[mt_encais]
>   FROM users a
> left join enca e on e.matr = a.matr
> left join encr c on (c.cod_encais = e.cod_encais)
>   where (YEAR(c.dat_enc) - c.annee > 3 ) and ((YEAR(getdate()) -
> YEAR(a.dat_nais)) between 54 and 70) and c.cod_nat = 'CN'
>   order by a.nom_, a.prn, c.annee asc;
>
> hello;
> my models are users, enca; encr;
> who can i performe this query using django ORM, note that i'm using MSSQL;
> thank very much
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/85f7f91b-0f52-4a96-8700-bee588eb4a3d%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAH9mneUNafffdDN0pE%2BCry_8-yGrYeWw1eziaRRe0znoskLmCQ%40mail.gmail.com.

Re: orm relation

[Integr@te System]

Hi issuer,

Plz refer to docs to customize statement base on your use case, my work for
a sample with limited time. Plz try remove space around equal sign.



On Tue, Dec 3, 2019, 19:26 bill dexter <55dexte...@gmail.com> wrote:

> hi friend;
> i tryed your answer, but it says that error in syntax near  __gt  in ""
> extractyear(dat_enc) - extractyear(annee)__gt = 3""; don't work
>
> Le lun. 2 déc. 2019 à 20:31, Integr@te System 
> a écrit :
>
>> Hi friend,
>>
>> All_list = list(users.object.aggregate(age =
>> cast(extractyear(datetime.now()) - extractyear(dat_nais), output_field =
>> IntField())).filter(extractyear(dat_enc) - extractyear(annee)__gt = 3 & 
>> extractyear(datetime.now())
>> - extractyear(dat_nais)__IN=range(54,70) & cod_nad__contain = 
>> 'CN').order_by('num___annee',
>> 'prn'))
>>
>>
>> Plz add more alias field as your sql command according to a sample above.
>>
>> Ps: check your question first(sql), then pass to communities.
>>
>>
>>
>> On Mon, Dec 2, 2019, 21:16 bill dexter <55dexte...@gmail.com> wrote:
>>
>>> thank you, but if you can give me answer to this  exacte probleme, as
>>> mentioned in question
>>>
>>> Le dim. 1 déc. 2019 à 18:55, Integr@te System 
>>> a écrit :
>>>
 Hi,

 here u can try for complex queryset



 https://docs.djangoproject.com/en/2.2/topics/db/queries/#complex-lookups-with-q

 On Sun, Dec 1, 2019, 21:08 bill dexter <55dexte...@gmail.com> wrote:

> ELECT a.matr
>   ,[nom]
>   ,[prn]
>   ,cast([dat_nais] as date) as dat_nais
>   , (YEAR(getdate()) - YEAR(dat_nais)) as age
>   ,cast([dat_deces] as date) as dat_deces
>   ,cast([dat_imm] as date) as dat_imm
>   ,cast([dat_aj] as date) as dat_j
>   ,[cod_position]
>   ,YEAR(dat_imm) as anne_imm
>   ,YEAR(dat_encais) as ann_regl
>   ,[annee]
>   ,[cod_nat]
>   ,cast([dat_e
>
> Snc] as date) as dat_regl
>   ,[mt_encais]
>   FROM users a
> left join enca e on e.matr = a.matr
> left join encr c on (c.cod_encais = e.cod_encais)
>   where (YEAR(c.dat_enc) - c.annee > 3 ) and ((YEAR(getdate()) -
> YEAR(a.dat_nais)) between 54 and 70) and c.cod_nat = 'CN'
>   order by a.nom_, a.prn, c.annee asc;
>
> hello;
> my models are users, enca; encr;
> who can i performe this query using django ORM, note that i'm using
> MSSQL; thank very much
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/85f7f91b-0f52-4a96-8700-bee588eb4a3d%40googlegroups.com
> 
> .
>
 --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-users/CAP5HUWqap3ddCEmW-7yD_uRWhwxDNQdSNZ%2BQPPvHzcAwDf1e5Q%40mail.gmail.com
 
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAA7%3DR20i-DNEFTgdHQ7bjmvzQHrmML3LjzamrSiK07oJgbC_%2Bw%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAP5HUWoRjYGOMbQf2Lz9npLi7%2BhsOfDoN-wYLAUAjbr9ites-Q%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegro

Re: orm relation

[bill dexter]

hi friend;
i tryed your answer, but it says that error in syntax near  __gt  in ""
extractyear(dat_enc) - extractyear(annee)__gt = 3""; don't work

Le lun. 2 déc. 2019 à 20:31, Integr@te System  a
écrit :

> Hi friend,
>
> All_list = list(users.object.aggregate(age =
> cast(extractyear(datetime.now()) - extractyear(dat_nais), output_field =
> IntField())).filter(extractyear(dat_enc) - extractyear(annee)__gt = 3 & 
> extractyear(datetime.now())
> - extractyear(dat_nais)__IN=range(54,70) & cod_nad__contain = 
> 'CN').order_by('num___annee',
> 'prn'))
>
>
> Plz add more alias field as your sql command according to a sample above.
>
> Ps: check your question first(sql), then pass to communities.
>
>
>
> On Mon, Dec 2, 2019, 21:16 bill dexter <55dexte...@gmail.com> wrote:
>
>> thank you, but if you can give me answer to this  exacte probleme, as
>> mentioned in question
>>
>> Le dim. 1 déc. 2019 à 18:55, Integr@te System 
>> a écrit :
>>
>>> Hi,
>>>
>>> here u can try for complex queryset
>>>
>>>
>>>
>>> https://docs.djangoproject.com/en/2.2/topics/db/queries/#complex-lookups-with-q
>>>
>>> On Sun, Dec 1, 2019, 21:08 bill dexter <55dexte...@gmail.com> wrote:
>>>
 ELECT a.matr
   ,[nom]
   ,[prn]
   ,cast([dat_nais] as date) as dat_nais
   , (YEAR(getdate()) - YEAR(dat_nais)) as age
   ,cast([dat_deces] as date) as dat_deces
   ,cast([dat_imm] as date) as dat_imm
   ,cast([dat_aj] as date) as dat_j
   ,[cod_position]
   ,YEAR(dat_imm) as anne_imm
   ,YEAR(dat_encais) as ann_regl
   ,[annee]
   ,[cod_nat]
   ,cast([dat_e

 Snc] as date) as dat_regl
   ,[mt_encais]
   FROM users a
 left join enca e on e.matr = a.matr
 left join encr c on (c.cod_encais = e.cod_encais)
   where (YEAR(c.dat_enc) - c.annee > 3 ) and ((YEAR(getdate()) -
 YEAR(a.dat_nais)) between 54 and 70) and c.cod_nat = 'CN'
   order by a.nom_, a.prn, c.annee asc;

 hello;
 my models are users, enca; encr;
 who can i performe this query using django ORM, note that i'm using
 MSSQL; thank very much

 --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-users/85f7f91b-0f52-4a96-8700-bee588eb4a3d%40googlegroups.com
 
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAP5HUWqap3ddCEmW-7yD_uRWhwxDNQdSNZ%2BQPPvHzcAwDf1e5Q%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAA7%3DR20i-DNEFTgdHQ7bjmvzQHrmML3LjzamrSiK07oJgbC_%2Bw%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAP5HUWoRjYGOMbQf2Lz9npLi7%2BhsOfDoN-wYLAUAjbr9ites-Q%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAA7%3DR22g%3DsVQJA1d4JdBC-biw%2BzcRCec9FPn_rbnBNo7%2B2%3DkTw%40mail.gmail.com.

Re: orm relation

[Integr@te System]

Hi friend,

All_list = list(users.object.aggregate(age =
cast(extractyear(datetime.now()) - extractyear(dat_nais), output_field =
IntField())).filter(extractyear(dat_enc) - extractyear(annee)__gt = 3
& extractyear(datetime.now())
- extractyear(dat_nais)__IN=range(54,70) & cod_nad__contain =
'CN').order_by('num___annee',
'prn'))


Plz add more alias field as your sql command according to a sample above.

Ps: check your question first(sql), then pass to communities.



On Mon, Dec 2, 2019, 21:16 bill dexter <55dexte...@gmail.com> wrote:

> thank you, but if you can give me answer to this  exacte probleme, as
> mentioned in question
>
> Le dim. 1 déc. 2019 à 18:55, Integr@te System 
> a écrit :
>
>> Hi,
>>
>> here u can try for complex queryset
>>
>>
>>
>> https://docs.djangoproject.com/en/2.2/topics/db/queries/#complex-lookups-with-q
>>
>> On Sun, Dec 1, 2019, 21:08 bill dexter <55dexte...@gmail.com> wrote:
>>
>>> ELECT a.matr
>>>   ,[nom]
>>>   ,[prn]
>>>   ,cast([dat_nais] as date) as dat_nais
>>>   , (YEAR(getdate()) - YEAR(dat_nais)) as age
>>>   ,cast([dat_deces] as date) as dat_deces
>>>   ,cast([dat_imm] as date) as dat_imm
>>>   ,cast([dat_aj] as date) as dat_j
>>>   ,[cod_position]
>>>   ,YEAR(dat_imm) as anne_imm
>>>   ,YEAR(dat_encais) as ann_regl
>>>   ,[annee]
>>>   ,[cod_nat]
>>>   ,cast([dat_e
>>>
>>> Snc] as date) as dat_regl
>>>   ,[mt_encais]
>>>   FROM users a
>>> left join enca e on e.matr = a.matr
>>> left join encr c on (c.cod_encais = e.cod_encais)
>>>   where (YEAR(c.dat_enc) - c.annee > 3 ) and ((YEAR(getdate()) -
>>> YEAR(a.dat_nais)) between 54 and 70) and c.cod_nat = 'CN'
>>>   order by a.nom_, a.prn, c.annee asc;
>>>
>>> hello;
>>> my models are users, enca; encr;
>>> who can i performe this query using django ORM, note that i'm using
>>> MSSQL; thank very much
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/85f7f91b-0f52-4a96-8700-bee588eb4a3d%40googlegroups.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAP5HUWqap3ddCEmW-7yD_uRWhwxDNQdSNZ%2BQPPvHzcAwDf1e5Q%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAA7%3DR20i-DNEFTgdHQ7bjmvzQHrmML3LjzamrSiK07oJgbC_%2Bw%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAP5HUWoRjYGOMbQf2Lz9npLi7%2BhsOfDoN-wYLAUAjbr9ites-Q%40mail.gmail.com.

Re: orm relation

[bill dexter]

thank you, but if you can give me answer to this  exacte probleme, as
mentioned in question

Le dim. 1 déc. 2019 à 18:55, Integr@te System  a
écrit :

> Hi,
>
> here u can try for complex queryset
>
>
>
> https://docs.djangoproject.com/en/2.2/topics/db/queries/#complex-lookups-with-q
>
> On Sun, Dec 1, 2019, 21:08 bill dexter <55dexte...@gmail.com> wrote:
>
>> ELECT a.matr
>>   ,[nom]
>>   ,[prn]
>>   ,cast([dat_nais] as date) as dat_nais
>>   , (YEAR(getdate()) - YEAR(dat_nais)) as age
>>   ,cast([dat_deces] as date) as dat_deces
>>   ,cast([dat_imm] as date) as dat_imm
>>   ,cast([dat_aj] as date) as dat_j
>>   ,[cod_position]
>>   ,YEAR(dat_imm) as anne_imm
>>   ,YEAR(dat_encais) as ann_regl
>>   ,[annee]
>>   ,[cod_nat]
>>   ,cast([dat_e
>>
>> Snc] as date) as dat_regl
>>   ,[mt_encais]
>>   FROM users a
>> left join enca e on e.matr = a.matr
>> left join encr c on (c.cod_encais = e.cod_encais)
>>   where (YEAR(c.dat_enc) - c.annee > 3 ) and ((YEAR(getdate()) -
>> YEAR(a.dat_nais)) between 54 and 70) and c.cod_nat = 'CN'
>>   order by a.nom_, a.prn, c.annee asc;
>>
>> hello;
>> my models are users, enca; encr;
>> who can i performe this query using django ORM, note that i'm using
>> MSSQL; thank very much
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/85f7f91b-0f52-4a96-8700-bee588eb4a3d%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAP5HUWqap3ddCEmW-7yD_uRWhwxDNQdSNZ%2BQPPvHzcAwDf1e5Q%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAA7%3DR20i-DNEFTgdHQ7bjmvzQHrmML3LjzamrSiK07oJgbC_%2Bw%40mail.gmail.com.

Re: orm relation

[Integr@te System]

Hi,

here u can try for complex queryset


https://docs.djangoproject.com/en/2.2/topics/db/queries/#complex-lookups-with-q

On Sun, Dec 1, 2019, 21:08 bill dexter <55dexte...@gmail.com> wrote:

> ELECT a.matr
>   ,[nom]
>   ,[prn]
>   ,cast([dat_nais] as date) as dat_nais
>   , (YEAR(getdate()) - YEAR(dat_nais)) as age
>   ,cast([dat_deces] as date) as dat_deces
>   ,cast([dat_imm] as date) as dat_imm
>   ,cast([dat_aj] as date) as dat_j
>   ,[cod_position]
>   ,YEAR(dat_imm) as anne_imm
>   ,YEAR(dat_encais) as ann_regl
>   ,[annee]
>   ,[cod_nat]
>   ,cast([dat_e
>
> Snc] as date) as dat_regl
>   ,[mt_encais]
>   FROM users a
> left join enca e on e.matr = a.matr
> left join encr c on (c.cod_encais = e.cod_encais)
>   where (YEAR(c.dat_enc) - c.annee > 3 ) and ((YEAR(getdate()) -
> YEAR(a.dat_nais)) between 54 and 70) and c.cod_nat = 'CN'
>   order by a.nom_, a.prn, c.annee asc;
>
> hello;
> my models are users, enca; encr;
> who can i performe this query using django ORM, note that i'm using MSSQL;
> thank very much
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/85f7f91b-0f52-4a96-8700-bee588eb4a3d%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAP5HUWqap3ddCEmW-7yD_uRWhwxDNQdSNZ%2BQPPvHzcAwDf1e5Q%40mail.gmail.com.

Re: ORM

[Jani Tiainen]

Most tools are capable of exporting uml to code format. A little googlin
should help.

ma 16. syysk. 2019 klo 18.57 Yann Mbella 
kirjoitti:

> I needed the contrary having uml diagrams and generating models from
> does diagram
>
> Le lun. 16 sept. 2019 à 16:43, Yann Mbella  a
> écrit :
>
>> Thanx for the information
>>
>> Le ven. 13 sept. 2019 à 06:24, Jani Tiainen  a écrit :
>>
>>> Hi.
>>>
>>> There exists django-extensions package that has management command to
>>> make UML diagram for your models.
>>>
>>>
>>> pe 13. syysk. 2019 klo 5.22 Yann Mbella 
>>> kirjoitti:
>>>
 Got little problem is it possible to have a plugin which mapps or
 reproduce my class diagrams drawn in an IDE of UML in my models app

 --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-users/13525e44-e176-409a-a55e-0595834c780f%40googlegroups.com
 
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAHn91odRe_26%3DuuAwCQOOapK_UeKPE1tOtmXQP-df8%3Dnxf4k3g%40mail.gmail.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAEaFv0FoE4D8_zNNv%2B3T3HUx1AzF%3DgGFPjUFLVL0sqQzc6Vtsg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAHn91odykqZaVd6jup5Di7R_P1Fp0%2BFS2UWDRrKVJfavZRWm0g%40mail.gmail.com.

Re: ORM

[Yann Mbella]

I needed the contrary having uml diagrams and generating models from
does diagram

Le lun. 16 sept. 2019 à 16:43, Yann Mbella  a
écrit :

> Thanx for the information
>
> Le ven. 13 sept. 2019 à 06:24, Jani Tiainen  a écrit :
>
>> Hi.
>>
>> There exists django-extensions package that has management command to
>> make UML diagram for your models.
>>
>>
>> pe 13. syysk. 2019 klo 5.22 Yann Mbella 
>> kirjoitti:
>>
>>> Got little problem is it possible to have a plugin which mapps or
>>> reproduce my class diagrams drawn in an IDE of UML in my models app
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/13525e44-e176-409a-a55e-0595834c780f%40googlegroups.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAHn91odRe_26%3DuuAwCQOOapK_UeKPE1tOtmXQP-df8%3Dnxf4k3g%40mail.gmail.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEaFv0FoE4D8_zNNv%2B3T3HUx1AzF%3DgGFPjUFLVL0sqQzc6Vtsg%40mail.gmail.com.

Re: ORM

[Yann Mbella]

Thanx for the information

Le ven. 13 sept. 2019 à 06:24, Jani Tiainen  a écrit :

> Hi.
>
> There exists django-extensions package that has management command to make
> UML diagram for your models.
>
>
> pe 13. syysk. 2019 klo 5.22 Yann Mbella 
> kirjoitti:
>
>> Got little problem is it possible to have a plugin which mapps or
>> reproduce my class diagrams drawn in an IDE of UML in my models app
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/13525e44-e176-409a-a55e-0595834c780f%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAHn91odRe_26%3DuuAwCQOOapK_UeKPE1tOtmXQP-df8%3Dnxf4k3g%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEaFv0GFdjA-W9XTWjG6as9KYy%3Dj-xy_PBMBG5v69fG9cvE%2B2Q%40mail.gmail.com.

Re: ORM

[Jani Tiainen]

Hi.

There exists django-extensions package that has management command to make
UML diagram for your models.


pe 13. syysk. 2019 klo 5.22 Yann Mbella  kirjoitti:

> Got little problem is it possible to have a plugin which mapps or
> reproduce my class diagrams drawn in an IDE of UML in my models app
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/13525e44-e176-409a-a55e-0595834c780f%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAHn91odRe_26%3DuuAwCQOOapK_UeKPE1tOtmXQP-df8%3Dnxf4k3g%40mail.gmail.com.

Re: Orm Query for this type of situation

[Devender Kumar]

Hi,
On this question only I took some step deep and able to find out one part
and need your help on the second half.
This requires a good understanding of Django annotation and subquery
features
If you have this thank for reading and helping me in advance
I am facing problem to filter calls on account.contact list which are
realted by contact phone no with and calls phone no
can any one help writing this query





On Thu, Jun 6, 2019 at 7:47 PM Devender Kumar 
wrote:

> Account /Lead Model
>  account_name
>  contact m2m realtion
>
> contact Model
> name
> phone No
>
> calls Model
> caller no # person phone no who is calling
> agent no #  person who is picking the call (CRM user)
> status 
> timestamp
> duration
>
> calls model is populated with some third party software and we are map its
> caller no and agent no their is no direct relationship
>
> I want to get the list of accounts in particular date range  with:
> total no of contact it have ,
> how many time its contact have called
> how many calls where missed inbound outbound etc
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Jun 6, 2019 at 4:49 PM Chetan Ganji 
> wrote:
>
>> Right now its an open ended question.
>>
>> If you post code for your models here, someone can help you.
>>
>> On Thu, Jun 6, 2019, 4:38 PM Devender Kumar  wrote:
>>
>>> Hi,
>>> I am working on CRM project and stuck with REPORTING part.
>>>
>>> I have Accounts , Lead , contact, Calls ,Concern modules
>>> account/Lead can have multiple contacts
>>> contacts have multiple calls and concerns
>>> calls have different status like Incoming outgoing missed call etc
>>>
>>> Question
>>> I need to create a reporting system
>>> In this I have to give a functionality like so
>>> list of accounts with counts of :
>>>  contacts, total calls,incoming, outgoing etc
>>>
>>>  how can list all this count for a given date ranges calls on all
>>> accounts
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/1cb5c003-7cbc-4d54-ae5b-21403e543b95%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAMKMUjtMaEncd6LF1J%2BmgKxA_NpL1FLFz5_UXi_JMLDD%3Du52Vw%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CALmCJr7ApqHCCsm73t%3DiuvzOYwE44Vh7pcdpH2bNkb-aH015fw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CALZ%3DbEL-FcgFqpjJtN9Up0JuOJ4WtO_fVKrPBERSYWKK%2B1r_gQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Orm Query for this type of situation

[Devender Kumar]

Account /Lead Model
 account_name
 contact m2m realtion

contact Model
name
phone No

calls Model
caller no # person phone no who is calling
agent no #  person who is picking the call (CRM user)
status 
timestamp
duration

calls model is populated with some third party software and we are map its
caller no and agent no their is no direct relationship

I want to get the list of accounts in particular date range  with:
total no of contact it have ,
how many time its contact have called
how many calls where missed inbound outbound etc














On Thu, Jun 6, 2019 at 4:49 PM Chetan Ganji  wrote:

> Right now its an open ended question.
>
> If you post code for your models here, someone can help you.
>
> On Thu, Jun 6, 2019, 4:38 PM Devender Kumar  wrote:
>
>> Hi,
>> I am working on CRM project and stuck with REPORTING part.
>>
>> I have Accounts , Lead , contact, Calls ,Concern modules
>> account/Lead can have multiple contacts
>> contacts have multiple calls and concerns
>> calls have different status like Incoming outgoing missed call etc
>>
>> Question
>> I need to create a reporting system
>> In this I have to give a functionality like so
>> list of accounts with counts of :
>>  contacts, total calls,incoming, outgoing etc
>>
>>  how can list all this count for a given date ranges calls on all
>> accounts
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/1cb5c003-7cbc-4d54-ae5b-21403e543b95%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAMKMUjtMaEncd6LF1J%2BmgKxA_NpL1FLFz5_UXi_JMLDD%3Du52Vw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CALmCJr7ApqHCCsm73t%3DiuvzOYwE44Vh7pcdpH2bNkb-aH015fw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Orm Query for this type of situation

[Chetan Ganji]

Right now its an open ended question.

If you post code for your models here, someone can help you.

On Thu, Jun 6, 2019, 4:38 PM Devender Kumar  wrote:

> Hi,
> I am working on CRM project and stuck with REPORTING part.
>
> I have Accounts , Lead , contact, Calls ,Concern modules
> account/Lead can have multiple contacts
> contacts have multiple calls and concerns
> calls have different status like Incoming outgoing missed call etc
>
> Question
> I need to create a reporting system
> In this I have to give a functionality like so
> list of accounts with counts of :
>  contacts, total calls,incoming, outgoing etc
>
>  how can list all this count for a given date ranges calls on all accounts
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/1cb5c003-7cbc-4d54-ae5b-21403e543b95%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAMKMUjtMaEncd6LF1J%2BmgKxA_NpL1FLFz5_UXi_JMLDD%3Du52Vw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM help with INNER JOIN and GROUP BY

[Kevin Jay]

You are correct.
Django creates a primary key of 'id' for each table as an integer by
default. So foreignkey will reference that primary key which by default is
an integer.
If the primary key is changed to some other type, Django can still handle
it.
Apparently using integers is the more efficient method.

On Mon, May 6, 2019 at 12:54 PM Matthew Pava  wrote:

> Well, I had always assumed that they do need to be integers, but it looks
> like this has been discussed at length.
>
> https://groups.google.com/forum/#!topic/django-users/0utRzn98Wxo
>
>
>
> It does seem clear, though, that the superior database design uses
> integers for ForeignKeys.
>
> And I didn’t find a way to make it clear to Django not to use an Integer
> ForeignKey.  Maybe something to do with the primary_key attribute on a
> CharField?
>
>
>
> *From:* django-users@googlegroups.com [mailto:
> django-users@googlegroups.com] *On Behalf Of *b...@tanners.org
> *Sent:* Monday, May 6, 2019 12:44 PM
> *To:* Django users
> *Subject:* Re: ORM help with INNER JOIN and GROUP BY
>
>
>
> Just want to make sure I understand. ForeighKeys need to be integers?
>
>
>
> Only integers?
>
>
>
>
>
> You need a ForeignKey relationship between the two models, which is an
> integer value, not a char. You’d have to do migrations to get this adjusted
> properly.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/9697de34-3d20-4023-81a7-8ea9f257f7f1%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/9697de34-3d20-4023-81a7-8ea9f257f7f1%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/27eee43ff8024c98bcf9a5e9cd4bfd89%40iss2.ISS.LOCAL
> <https://groups.google.com/d/msgid/django-users/27eee43ff8024c98bcf9a5e9cd4bfd89%40iss2.ISS.LOCAL?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEtabwdPi16kCzZAx6btpyRFGVn0JyaXGjL%2Bw8JTj2u6pHZxBg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM help with INNER JOIN and GROUP BY

[bob]

class Monolithic(models.Model):
facepng_id = models.IntegerField(blank=True, null=True)


facepng_id as Integer not ForeignKey?


> UPDATE items_monolithic SET facepng_id=items_monolithic.id FROM 
items_monolithic INNER JOIN items_facepng ON 
items_monolithic.object=items_facepng.obj ;
Error: near "from": syntax error
 
I'll keep working on the code.

Add a facepng_id Integer Field to Monolith. Run makemigrations. Add a 
> RunSQL command to the new migrations file (
> https://docs.djangoproject.com/en/2.2/ref/migration-operations/#runsql).  
> Something like this:
>
> Update items_monolith SET facepng_id=items_monolith.id FROM 
> items_monolith INNER JOIN items_facepng ON 
> items_monolith.object=items_facepng.obj 
>
>  
>
> Then delete the object and obj fields from the models. Run makemigrations 
> again. Verify that your app doesn’t reference those fields anywhere else.
>
>  
>
> Then you could access facepng like so:
>
> monolithic.facepng_set
>
>  
>
> Of course, this is just a very rough idea of what to do, and I’m not sure 
> what the whole structure of your tables is.
>
>  
>
> *From:* django...@googlegroups.com  [mailto:
> django...@googlegroups.com ] *On Behalf Of *b...@tanners.org 
> 
> *Sent:* Monday, May 6, 2019 11:28 AM
> *To:* Django users
> *Subject:* ORM help with INNER JOIN and GROUP BY
>
>  
>
> I've inherited an application written django 1.11. The old application 
> uses raw() with INNER JOIN and GROUP BY. 
>
>  I cannot seem to figure out how to do inner join and group by properly 
> the ORM way.
>
>  
>
> The raw() query is below.
>
>  
>
> SELECT  * FROM items_monolithic
>
> INNER JOIN items_facepng
>
> ON items_monolithic.object == items_facepng.obj
>
> GROUP BY items_monolithic.object
>
> ORDER BY object ASC
>
>  
>
>  
>
> Things kind of work with raw() but that doesn't feel right. And I get 
> nasty warnings about RawQuerySet not supporting certain things when I try 
> to use the query set that is returned.
>
>  
>
> From what I understand every monolithic object has 1 or more faces 
> (graphic/picture).
>
>  
>
> I would call the monolithic a one-to-many relationship with facepng but I 
> see django calls this a ForeignKey.
>
>  
>
> I'm working with these models (yes, a field named object is "bad", it's 
> what I was given)
>
>  
>
>  
>
> class Monolithic(models.Model):
>
>object = models.CharField(max_length=128, blank=False, null=False, 
> unique=True)
>
>  
>
> class FacePng(models.Model):
>
> obj = models.CharField(max_length=128, blank=True, null=True)
>
>  
>
>  
>
> I do not see the ForeignKey relationship between Monolithic and FacePng. 
>
>  
>
> Changing Monolithic class to models.ForeignKey() breaks lots of things.
>
>  
>
> So I'd prefer to figure out how to do the inner join and group by query 
> but if that's not the django way and I need to change Monolithic.objects to 
> a ForeignKey() and fix all the stuff that is broken I can do that that too. 
>
>  
>
> Just need some guidance on how to proceed.
>
>  
>
>  
>
>  
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django...@googlegroups.com .
> To post to this group, send email to djang...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/ab7ecf5b-9ae8-4428-9502-6b7d5dec03b5%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/7af29fa6-5cfc-4bda-a471-ed8ee26b37ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM help with INNER JOIN and GROUP BY

[Kevin Jay]

 Maybe that was a typo?
The foreignkey relationship would look like this:
object = models.ForeignKey(FacePng'', max_length=128, blank=False,
null=False, unique=True)

On Mon, May 6, 2019 at 12:44 PM  wrote:

> Just want to make sure I understand. ForeighKeys need to be integers?
>
> Only integers?
>
>
> You need a ForeignKey relationship between the two models, which is an
>> integer value, not a char. You’d have to do migrations to get this adjusted
>> properly.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/9697de34-3d20-4023-81a7-8ea9f257f7f1%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEtabwfknJFSeHiYLcLdQ6-0soywHaHwsPz8SLm_T24tbT0Hpg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

RE: ORM help with INNER JOIN and GROUP BY

[Matthew Pava]

Well, I had always assumed that they do need to be integers, but it looks like 
this has been discussed at length.
https://groups.google.com/forum/#!topic/django-users/0utRzn98Wxo

It does seem clear, though, that the superior database design uses integers for 
ForeignKeys.
And I didn’t find a way to make it clear to Django not to use an Integer 
ForeignKey.  Maybe something to do with the primary_key attribute on a 
CharField?

From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of b...@tanners.org
Sent: Monday, May 6, 2019 12:44 PM
To: Django users
Subject: Re: ORM help with INNER JOIN and GROUP BY

Just want to make sure I understand. ForeighKeys need to be integers?

Only integers?


You need a ForeignKey relationship between the two models, which is an integer 
value, not a char. You’d have to do migrations to get this adjusted properly.
--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com<mailto:django-users+unsubscr...@googlegroups.com>.
To post to this group, send email to 
django-users@googlegroups.com<mailto:django-users@googlegroups.com>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/9697de34-3d20-4023-81a7-8ea9f257f7f1%40googlegroups.com<https://groups.google.com/d/msgid/django-users/9697de34-3d20-4023-81a7-8ea9f257f7f1%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/27eee43ff8024c98bcf9a5e9cd4bfd89%40iss2.ISS.LOCAL.
For more options, visit https://groups.google.com/d/optout.

Re: ORM help with INNER JOIN and GROUP BY

[bob]

Just want to make sure I understand. ForeighKeys need to be integers?

Only integers?


You need a ForeignKey relationship between the two models, which is an 
> integer value, not a char. You’d have to do migrations to get this adjusted 
> properly.  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/9697de34-3d20-4023-81a7-8ea9f257f7f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM help with INNER JOIN and GROUP BY

[Kevin Jay]

The field within your class should be set to ForeignKey.


Try this:
Object = models.ForeignKey(‘FacePng’, on_delete=SET_NULL, max_length= some_int)

on_delete and max_length need to be set based on your requirements 

Sent from my iPhone

> On May 6, 2019, at 11:28 AM, b...@tanners.org wrote:
> 
> I've inherited an application written django 1.11. The old application uses 
> raw() with INNER JOIN and GROUP BY. 
>  I cannot seem to figure out how to do inner join and group by properly the 
> ORM way.
> 
> The raw() query is below.
> 
> SELECT  * FROM items_monolithic
> INNER JOIN items_facepng
> ON items_monolithic.object == items_facepng.obj
> GROUP BY items_monolithic.object
> ORDER BY object ASC
> 
> 
> Things kind of work with raw() but that doesn't feel right. And I get nasty 
> warnings about RawQuerySet not supporting certain things when I try to use 
> the query set that is returned.
> 
> From what I understand every monolithic object has 1 or more faces 
> (graphic/picture).
> 
> I would call the monolithic a one-to-many relationship with facepng but I see 
> django calls this a ForeignKey.
> 
> I'm working with these models (yes, a field named object is "bad", it's what 
> I was given)
> 
> 
> class Monolithic(models.Model):
> object = models.CharField(max_length=128, blank=False, null=False, 
> unique=True)
> 
> class FacePng(models.Model):
> obj = models.CharField(max_length=128, blank=True, null=True)
> 
> 
> I do not see the ForeignKey relationship between Monolithic and FacePng. 
> 
> Changing Monolithic class to models.ForeignKey() breaks lots of things.
> 
> So I'd prefer to figure out how to do the inner join and group by query but 
> if that's not the django way and I need to change Monolithic.objects to a 
> ForeignKey() and fix all the stuff that is broken I can do that that too. 
> 
> Just need some guidance on how to proceed.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/ab7ecf5b-9ae8-4428-9502-6b7d5dec03b5%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/D5B5695B-D057-4D4D-8F53-2BAF1D6B666A%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

RE: ORM help with INNER JOIN and GROUP BY

[Matthew Pava]

That design could definitely be improved, and it will have to be in order to 
use the ORM effectively. Then you’d have to change every reference to the 
fields in all the raw querysets in the app.

You need a ForeignKey relationship between the two models, which is an integer 
value, not a char. You’d have to do migrations to get this adjusted properly.

Add a facepng_id Integer Field to Monolith. Run makemigrations. Add a RunSQL 
command to the new migrations file 
(https://docs.djangoproject.com/en/2.2/ref/migration-operations/#runsql).  
Something like this:
Update items_monolith SET facepng_id=items_monolith.id FROM items_monolith 
INNER JOIN items_facepng ON items_monolith.object=items_facepng.obj

Then delete the object and obj fields from the models. Run makemigrations 
again. Verify that your app doesn’t reference those fields anywhere else.

Then you could access facepng like so:
monolithic.facepng_set

Of course, this is just a very rough idea of what to do, and I’m not sure what 
the whole structure of your tables is.

From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of b...@tanners.org
Sent: Monday, May 6, 2019 11:28 AM
To: Django users
Subject: ORM help with INNER JOIN and GROUP BY

I've inherited an application written django 1.11. The old application uses 
raw() with INNER JOIN and GROUP BY.
 I cannot seem to figure out how to do inner join and group by properly the ORM 
way.

The raw() query is below.

SELECT  * FROM items_monolithic
INNER JOIN items_facepng
ON items_monolithic.object == items_facepng.obj
GROUP BY items_monolithic.object
ORDER BY object ASC


Things kind of work with raw() but that doesn't feel right. And I get nasty 
warnings about RawQuerySet not supporting certain things when I try to use the 
query set that is returned.

>From what I understand every monolithic object has 1 or more faces 
>(graphic/picture).

I would call the monolithic a one-to-many relationship with facepng but I see 
django calls this a ForeignKey.

I'm working with these models (yes, a field named object is "bad", it's what I 
was given)


class Monolithic(models.Model):
   object = models.CharField(max_length=128, blank=False, null=False, 
unique=True)

class FacePng(models.Model):
obj = models.CharField(max_length=128, blank=True, null=True)


I do not see the ForeignKey relationship between Monolithic and FacePng.

Changing Monolithic class to models.ForeignKey() breaks lots of things.

So I'd prefer to figure out how to do the inner join and group by query but if 
that's not the django way and I need to change Monolithic.objects to a 
ForeignKey() and fix all the stuff that is broken I can do that that too.

Just need some guidance on how to proceed.



--
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to 
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ab7ecf5b-9ae8-4428-9502-6b7d5dec03b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/573f67effc814147bb18f8e4327d2c23%40iss2.ISS.LOCAL.
For more options, visit https://groups.google.com/d/optout.

Re: ORM - Subquery / Window

[marco . nicotra]

As suggested i'm writing what i've already tried:

Raw SQL using the query written in the first message, selecting only the id 
field and passing it to the ORM as id__in=ids. Slow as hell, unusable.

Declared a WIndow function to use as filter: 
Article.objects.annotate(max_rev=Window(expression=Max("revision"), 
partition_by=F("pn"))).filter(revision=F("max_rev"))
But Django complained that i cannot use a window function in a where clause 
(that's correct).

Then i've tried to use the window as subquery:

window_query = Article.objects.annotate(max_rev=Window(expression=Max(
"revision"), partition_by=F("pn")))
result = Article.objects.filter(revision= Subquery(window_query)

I've tried also with OuterRef, to use the max_rev annotation as a join, no 
luck.
I'm out of ideas!


Il giorno giovedì 7 giugno 2018 14:44:41 UTC+2, marco@gmail.com ha 
scritto:
>
> Hello everyone,
>
> i want to extract the records with Max("revision") from a table like this:
>
> pn1rev1description
> pn1rev2description
> pn2rev1anotherdescription
> pn1rev3description
> pn2rev2anotherdescription
>
> The first column is a part number, the second is its revision index (which 
> is created every time the part number is modifyied).
> That is quite easy in pure SQL:
>
> SELECT
>   id,
>   pn,
>   revision,
>   description
> FROM (SELECT
> id,
> pn,
> revision,
> MAX(revision)
> OVER (
>   PARTITION BY pn ) max_rev,
> description
>   FROM en_articles) maxart
> WHERE revision = max_rev;
>
> I cannot understand how to do the same with Django's ORM, i've tried every 
> combination of Subquery/Window without getting anywhere.
> Does anyone know how to do it?
>
> Thanks in advance
> Marco
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/104dab1e-c12c-47f7-8dbc-3c2a1931aaeb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ORM Foreign Keys][Performance] How to use Charfield Foreign Keys efficiently?

[Peter of the Norse]

This should be a bug.  Most databases don’t even allow foreign keys to be a 
different type.  I didn’t see anything in the docs about this, but it should 
happen automatically.  There are several examples of using non-int fields, e.g. 
UUID.  Can you provide your migration file for OmEvent and model definition for 
CaseInfo2?

- Peter of the Norse

> On Nov 21, 2017, at 7:16 AM, Dominik Szmaj  wrote:
> 
> Hey,
> 
> 'case_id' is a monkeypatch number(*) field so it works quick for now, not a 
> pk, but unique:
> class OmEvent(Model):
> case = ForeignKey('CaseInfo2', to_field='case_id', related_name='events', 
> blank=True, null=True)
> 
> class Meta:
> db_table = 'om_event'
> 
> old form:
> class OmEvent(Model):
> case = ForeignKey('CaseInfo2', related_name='events', blank=True, 
> null=True)
> 
> class Meta:
> db_table = 'om_event'
> which points to 'case_number' pk which is a varchar.
> 
> Everything is managed by db router which prevents migrations on remote db.
> 
> Regards,
> Dominik
> 
> W dniu wtorek, 21 listopada 2017 15:00:49 UTC+1 użytkownik Avraham Serour 
> napisał:
>> 
>> can you post your model?
>> 
>>> On Tue, Nov 21, 2017 at 3:31 PM, Dominik Szmaj  wrote:
>>> Hey,
>>> 
>>> I have a very big performance problem with Django and Oracle db.
>>> 
>>> This legacy db has lots of primary keys as strings from the time when those 
>>> id's were alphanumerical so it can't be simply converted to number.
>>> 
>>> So when I set FK on such varchar column django seems to not crash but it 
>>> converts to int and search whole table of strings with a number which takes 
>>> ages to complete.
>>> 
>>> Is there maybe some sane solution to the problem? Why it forces me to use 
>>> numbers on varchar FK?
>>> 
>>> Best regards.
>>> Dominik
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to django-users...@googlegroups.com.
>>> To post to this group, send email to django...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-users.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-users/e4e51608-740c-410c-bc47-21d189c0edf6%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
> 
> W dniu wtorek, 21 listopada 2017 15:00:49 UTC+1 użytkownik Avraham Serour 
> napisał:
>> 
>> can you post your model?
>> 
>>> On Tue, Nov 21, 2017 at 3:31 PM, Dominik Szmaj  wrote:
>>> Hey,
>>> 
>>> I have a very big performance problem with Django and Oracle db.
>>> 
>>> This legacy db has lots of primary keys as strings from the time when those 
>>> id's were alphanumerical so it can't be simply converted to number.
>>> 
>>> So when I set FK on such varchar column django seems to not crash but it 
>>> converts to int and search whole table of strings with a number which takes 
>>> ages to complete.
>>> 
>>> Is there maybe some sane solution to the problem? Why it forces me to use 
>>> numbers on varchar FK?
>>> 
>>> Best regards.
>>> Dominik
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to django-users...@googlegroups.com.
>>> To post to this group, send email to django...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-users.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-users/e4e51608-740c-410c-bc47-21d189c0edf6%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/a8512002-4fac-4ae0-b6ee-35a239d2670f%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/C2CCC001-3BEE-411B-9669-5D66C4A4DFEB%40Radio1190.org.
For more options, visit https://groups.google.com/d/optout.

Re: [ORM Foreign Keys][Performance] How to use Charfield Foreign Keys efficiently?

[Dominik Szmaj]

Hey,

'case_id' is a monkeypatch number(*) field so it works quick for now, not a 
pk, but unique:

class OmEvent(Model):
case = ForeignKey('CaseInfo2', to_field='case_id', related_name='events', 
blank=True, null=True)

class Meta:
db_table = 'om_event'


old form:

class OmEvent(Model):
case = ForeignKey('CaseInfo2', related_name='events', blank=True, null=True)

class Meta:
db_table = 'om_event'

which points to 'case_number' pk which is a varchar.

Everything is managed by db router which prevents migrations on remote db.

Regards,
Dominik

W dniu wtorek, 21 listopada 2017 15:00:49 UTC+1 użytkownik Avraham Serour 
napisał:
>
> can you post your model?
>
> On Tue, Nov 21, 2017 at 3:31 PM, Dominik Szmaj  > wrote:
>
>> Hey,
>>
>> I have a very big performance problem with Django and Oracle db.
>>
>> This legacy db has lots of primary keys as strings from the time when 
>> those id's were alphanumerical so it can't be simply converted to number.
>>
>> So when I set FK on such varchar column django seems to not crash but it 
>> converts to int and search whole table of strings with a number which takes 
>> ages to complete.
>>
>> Is there maybe some sane solution to the problem? Why it forces me to use 
>> numbers on varchar FK?
>>
>> Best regards.
>> Dominik
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/e4e51608-740c-410c-bc47-21d189c0edf6%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
W dniu wtorek, 21 listopada 2017 15:00:49 UTC+1 użytkownik Avraham Serour 
napisał:
>
> can you post your model?
>
> On Tue, Nov 21, 2017 at 3:31 PM, Dominik Szmaj  > wrote:
>
>> Hey,
>>
>> I have a very big performance problem with Django and Oracle db.
>>
>> This legacy db has lots of primary keys as strings from the time when 
>> those id's were alphanumerical so it can't be simply converted to number.
>>
>> So when I set FK on such varchar column django seems to not crash but it 
>> converts to int and search whole table of strings with a number which takes 
>> ages to complete.
>>
>> Is there maybe some sane solution to the problem? Why it forces me to use 
>> numbers on varchar FK?
>>
>> Best regards.
>> Dominik
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/e4e51608-740c-410c-bc47-21d189c0edf6%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/a8512002-4fac-4ae0-b6ee-35a239d2670f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ORM Foreign Keys][Performance] How to use Charfield Foreign Keys efficiently?

[Jani Tiainen]

Please, post your models in question. It's probably something really 
simple to resolve in general.



On 21.11.2017 15.59, Avraham Serour wrote:

can you post your model?

On Tue, Nov 21, 2017 at 3:31 PM, Dominik Szmaj 
mailto:dominik.sz...@snello.pl>> wrote:


Hey,

I have a very big performance problem with Django and Oracle db.

This legacy db has lots of primary keys as strings from the time
when those id's were alphanumerical so it can't be simply
converted to number.

So when I set FK on such varchar column django seems to not crash
but it converts to int and search whole table of strings with a
number which takes ages to complete.

Is there maybe some sane solution to the problem? Why it forces me
to use numbers on varchar FK?

Best regards.
Dominik
-- 
You received this message because you are subscribed to the Google

Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-users+unsubscr...@googlegroups.com
.
To post to this group, send email to django-users@googlegroups.com
.
Visit this group at https://groups.google.com/group/django-users
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/e4e51608-740c-410c-bc47-21d189c0edf6%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout
.


--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-users+unsubscr...@googlegroups.com 
.
To post to this group, send email to django-users@googlegroups.com 
.

Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6tKd-c9bUn5qSv9hFQvPSvVVyqOi1GpOWYQWz2uzL52h%2Bg%40mail.gmail.com 
.

For more options, visit https://groups.google.com/d/optout.


--
Jani Tiainen

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/39b6207f-6524-a6f3-af51-476e689ea25f%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ORM Foreign Keys][Performance] How to use Charfield Foreign Keys efficiently?

[Avraham Serour]

can you post your model?

On Tue, Nov 21, 2017 at 3:31 PM, Dominik Szmaj 
wrote:

> Hey,
>
> I have a very big performance problem with Django and Oracle db.
>
> This legacy db has lots of primary keys as strings from the time when
> those id's were alphanumerical so it can't be simply converted to number.
>
> So when I set FK on such varchar column django seems to not crash but it
> converts to int and search whole table of strings with a number which takes
> ages to complete.
>
> Is there maybe some sane solution to the problem? Why it forces me to use
> numbers on varchar FK?
>
> Best regards.
> Dominik
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/e4e51608-740c-410c-bc47-21d189c0edf6%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6tKd-c9bUn5qSv9hFQvPSvVVyqOi1GpOWYQWz2uzL52h%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[guettli]

Thank you for sharing. 

Yes, the model is not difficult, that's why everybody helps himself up to 
now. Maybe this is the best solution and there is not much in common.

The things which are in common (ORM, Admin-Interface) are already provided 
by django.

It's just a bit of glue-code to get it working.

Again, thank you for sharing your solution.

Regards,
  Thomas

Am Samstag, 8. April 2017 18:58:59 UTC+2 schrieb Scot Hacker:
>
> On Tuesday, April 4, 2017 at 5:18:42 AM UTC-7, guettli wrote:
>>
>> In the past I was told: Don't store logs in the database.
>>
>
> For general purposes, I agree with this. Logging is a python standard, 
> logs can be verbose, logrolling solutions are well established (and built 
> in), etc. However, there are certain situations or activities where 
> database logging makes sense, most likely *in addition to* standard logging 
> rather than instead of. In one of my projects, half a dozen non-technical 
> managers need the ability to track certain types of actions (related to 
> account activations at a school). For this, I developed a simple ORM 
> logging solution that lets those managers search and filter these special 
> logs in the Django admin. 
>
> It's not something that really deserves to be its own project, IMO - just 
> a typical thing a dev might do in Django to satisfy an institutional need. 
> But I've put my solution in this gist, in case its helpful:
>
> https://gist.github.com/shacker/05bc1de527a2d7412de361ac659aecde
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/e20e56bd-e82c-4027-afa2-bc08ca35c00b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[Scot Hacker]

On Tuesday, April 4, 2017 at 5:18:42 AM UTC-7, guettli wrote:
>
> In the past I was told: Don't store logs in the database.
>

For general purposes, I agree with this. Logging is a python standard, logs 
can be verbose, logrolling solutions are well established (and built in), 
etc. However, there are certain situations or activities where database 
logging makes sense, most likely *in addition to* standard logging rather 
than instead of. In one of my projects, half a dozen non-technical managers 
need the ability to track certain types of actions (related to account 
activations at a school). For this, I developed a simple ORM logging 
solution that lets those managers search and filter these special logs in 
the Django admin. 

It's not something that really deserves to be its own project, IMO - just a 
typical thing a dev might do in Django to satisfy an institutional need. 
But I've put my solution in this gist, in case its helpful:

https://gist.github.com/shacker/05bc1de527a2d7412de361ac659aecde
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/c9486b92-010d-445b-b137-1b18f7ca5efb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[guettli]

Hi Marten,

Am Donnerstag, 6. April 2017 14:10:58 UTC+2 schrieb knbk:
>
> Hi Thomas,
>
> The primary purpose of logging is to catch and examine errors. If 
> something went wrong, you want to know *when *and *why*. Logging to a 
> database increases the complexity and greatly increases the number of 
> things that can go wrong. The last thing you want to find out when 
> retracing an error is that you don't have any logs because the logging 
> system failed. You may also need to log errors that happened during 
> startup, before a database connection can be established. Logging to file 
> is the simplest method, and has the least chance of failure. That's why you 
> should always log to file. 
>
> The two options are not mutually exclusive. Like you said, times have 
> changed, and the overhead to store logs both in a file and in a database 
> are nowadays acceptable. If you have a good reason to store the logs in a 
> database, then go ahead. Just remember that it should be *in addition to 
> *file-based 
> logging. 
>
>
Yes, you are right. During the initialization of processes no db 
connections exists yet. I don't like redundancy but here its needed for a 
higher availability. 

Thank you for your reply.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/a9a0fe4e-4b65-4ceb-8336-cc4ceeb82f2c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[guettli]

Am Donnerstag, 6. April 2017 10:42:17 UTC+2 schrieb Christian Ledermann:
>
> On 6 April 2017 at 09:15, guettli > 
> wrote: 
> > Hello Brick Wall, how are you doing? 
>
> Hello Stonemason. 
>
> What is your question? 
>
>

It was more an idea than a question.

The question could be: What do you think about this idea?

I am lazy and like postgresql. Up to now we store are logs in files and we 
search for an alternative.

I am unfamiliar with ElasticSearch (DB of ELK-Stack).

ELK-Stack is very popular, but maybe is overrated. I don't know. Why not 
use PostgreSQL?



 

> I do not have a strong opinion on your approach - i don't even know 
> the problem you are trying to solve. 
> or how big your logs are. a couple of KB per day or some GB per hour? 
>
>
Traffic does not play any role here. This question is only about the 
structure (tables and columns), not about the rows.

 

> [the brickwall shrugs its shoulders] 
>
>

Thank you for your reply.


 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/cc315693-bd7b-47a0-bb54-c83a822bf4a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[knbk]

Hi Thomas,

The primary purpose of logging is to catch and examine errors. If something 
went wrong, you want to know *when *and *why*. Logging to a database 
increases the complexity and greatly increases the number of things that 
can go wrong. The last thing you want to find out when retracing an error 
is that you don't have any logs because the logging system failed. You may 
also need to log errors that happened during startup, before a database 
connection can be established. Logging to file is the simplest method, and 
has the least chance of failure. That's why you should always log to file. 

The two options are not mutually exclusive. Like you said, times have 
changed, and the overhead to store logs both in a file and in a database 
are nowadays acceptable. If you have a good reason to store the logs in a 
database, then go ahead. Just remember that it should be *in addition to 
*file-based 
logging. 

Kind regards,

Marten

On Tuesday, April 4, 2017 at 2:18:42 PM UTC+2, guettli wrote:
>
> In the past I was told: Don't store logs in the database.
>
> Time (and hardware) has changed.
>
> I think it is time to store logs where I have great tools for analyzing 
> logs.
>
> Storing in a model which I can access via django orm is my current 
> strategy.
>
> It seems no one has done this before. I could not find such a project up 
> to now.
>
> My first draft looks like this:
>
>
> class Log(models.Model):
> datetime=models.DateTimeField(default=datetime.datetime.now, 
> db_index=True)
> data=jsonfield.JSONField()
> host=models.CharField(max_length=256, default='localhost', db_index=True)
> system=models.CharField(max_length=256, default='', db_index=True)
>
> I am missing two things:
>
>  Missing1: Log level: INFO, WARN, ...
>
>  Missing2: A way to store exceptions.
>
>
> What do you think?
>
> What's wrong with this, what could be improved?
>
> Regards,
>   Thomas Güttler
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/21746446-93c7-40d7-b2fa-6adaad9f4aee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[Christian Ledermann]

On 6 April 2017 at 09:15, guettli  wrote:
> Hello Brick Wall, how are you doing?

Hello Stonemason.

What is your question?

I do not have a strong opinion on your approach - i don't even know
the problem you are trying to solve.
or how big your logs are. a couple of KB per day or some GB per hour?

[the brickwall shrugs its shoulders]

>
> Am Dienstag, 4. April 2017 14:18:42 UTC+2 schrieb guettli:
>>
>> In the past I was told: Don't store logs in the database.
>>
>> Time (and hardware) has changed.
>>
>> I think it is time to store logs where I have great tools for analyzing
>> logs.
>>
>> Storing in a model which I can access via django orm is my current
>> strategy.
>>
>> It seems no one has done this before. I could not find such a project up
>> to now.
>>
>> My first draft looks like this:
>>
>>
>> class Log(models.Model):
>> datetime=models.DateTimeField(default=datetime.datetime.now,
>> db_index=True)
>> data=jsonfield.JSONField()
>> host=models.CharField(max_length=256, default='localhost',
>> db_index=True)
>> system=models.CharField(max_length=256, default='', db_index=True)
>>
>> I am missing two things:
>>
>>  Missing1: Log level: INFO, WARN, ...
>>
>>  Missing2: A way to store exceptions.
>>
>>
>> What do you think?
>>
>> What's wrong with this, what could be improved?
>>
>> Regards,
>>   Thomas Güttler
>>
>>
>>
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/c8742484-a252-4c05-939a-f7c3f2b4f505%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



-- 
Best Regards,

Christian Ledermann

Newark-on-Trent - UK
Mobile : +44 7474997517

https://uk.linkedin.com/in/christianledermann
https://github.com/cleder/


<*)))>{

If you save the living environment, the biodiversity that we have left,
you will also automatically save the physical environment, too. But If
you only save the physical environment, you will ultimately lose both.

1) Don’t drive species to extinction

2) Don’t destroy a habitat that species rely on.

3) Don’t change the climate in ways that will result in the above.

}<(((*>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CABCjzWr3G%3DNnqkx75CEEXZ3bMXsV9Wd31A2vhMAxytjJREEBEw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM for structured Logs

[guettli]

Hello Brick Wall, how are you doing?

Am Dienstag, 4. April 2017 14:18:42 UTC+2 schrieb guettli:
>
> In the past I was told: Don't store logs in the database.
>
> Time (and hardware) has changed.
>
> I think it is time to store logs where I have great tools for analyzing 
> logs.
>
> Storing in a model which I can access via django orm is my current 
> strategy.
>
> It seems no one has done this before. I could not find such a project up 
> to now.
>
> My first draft looks like this:
>
>
> class Log(models.Model):
> datetime=models.DateTimeField(default=datetime.datetime.now, 
> db_index=True)
> data=jsonfield.JSONField()
> host=models.CharField(max_length=256, default='localhost', db_index=True)
> system=models.CharField(max_length=256, default='', db_index=True)
>
> I am missing two things:
>
>  Missing1: Log level: INFO, WARN, ...
>
>  Missing2: A way to store exceptions.
>
>
> What do you think?
>
> What's wrong with this, what could be improved?
>
> Regards,
>   Thomas Güttler
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/c8742484-a252-4c05-939a-f7c3f2b4f505%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM PostgreSQL stuff

[Avraham Serour]

I didn't quite understood the question, you want to find the division of
two prices for each item, but each item has only one price according to the
model you posted.

can you express what you want to accomplish in python or SQL?

On Sat, Nov 26, 2016 at 10:15 PM, Artem Bernatskyy <
artem.bernats...@gmail.com> wrote:

>
> Hello
>
> Is it possible to do in ORM or in PostgreSQL
>
> Assume we have a table with date and price
>
> # class Item(models.Model):
> #price = models.DecimalField()
> #date = models.DateField()
>
> We need for every item in table find division of two prices by two dates
> and then order_by that result of division
>
>
> P.S. i can accomplish this via signals but... i don't want to create no
> more denormalized tables
>
> Tnx in advance :)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/b1b53243-1ba6-4ff9-8493-35519620b3f4%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6tJsuf%3Ds8QhbA1W81_mkZ%2BReBt1tm8n%3D6XjCSTkbMHL08A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Query question

[Joss Ingram]

Ok, thanks.

On Monday, 13 July 2015 16:14:29 UTC+1, Avraham Serour wrote:
>
> Personally I don't see obvious errors, but it is hard to tell with only 
> this small snippet.
>
> If it works what are you worried about? Performance? If you are having 
> performance problems you can try caching the query
>
> On Mon, Jul 13, 2015 at 6:08 PM, Joss Ingram  > wrote:
>
>> Ok, thanks fair enough.
>>
>> my Question then is :
>>
>> Is the query above bad?
>>
>> On Mon, Jul 13, 2015 at 4:00 PM, Avraham Serour > > wrote:
>>
>>> You told a story but didn't ask a question
>>>
>>> On Mon, Jul 13, 2015 at 5:54 PM, Joss Ingram >> > wrote:
>>>
 I'm not getting much joy with my question, is there too little info? 

 On Friday, 10 July 2015 15:31:50 UTC+1, Joss Ingram wrote:
>
> Hi, 
>
> In my models I have an event index page type, which can have child 
> event pages, each event page can be tagged (using taggit), and I want to 
> produce a list of distinct tags used in the events that belong to that 
> event index on the index itself. I'm using the Django CMS Wagtail, but I 
> guess this is really a Django ORM query question. I''ve got some code as 
> part of my model which is working but I don't think it's the most 
> efficient 
> way of doing this. Still trying unlearn doing things with SQL and learn 
> how 
> the same thing should be done in Django.
>
> My code as part of the event index class is 
>
> @property
> def event_tag_list(self):
> child_events = EventPage.objects.live().descendant_of(self)
> return 
> Tag.objects.filter(mysite_eventpagetag_items__isnull=False).filter(mysite_eventpagetag_items__content_object_id=child_events).order_by('slug').distinct('slug')
>
> Any feedback would be appreciated.
>
> Thanks
>
> Joss
>
> 
>
>  -- 
 You received this message because you are subscribed to the Google 
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to django-users...@googlegroups.com .
 To post to this group, send email to django...@googlegroups.com 
 .
 Visit this group at http://groups.google.com/group/django-users.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-users/61cf7c97-ae40-4c71-a094-ff1d819a7ce5%40googlegroups.com
  
 
 .

 For more options, visit https://groups.google.com/d/optout.

>>>
>>>  -- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "Django users" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/django-users/qB8W-I3jJWg/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to 
>>> django-users...@googlegroups.com .
>>> To post to this group, send email to django...@googlegroups.com 
>>> .
>>> Visit this group at http://groups.google.com/group/django-users.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-users/CAFWa6t%2B1dpT%3D3oYG%2BqRopr9vWWXwY573%2BM%2BzkuT8ijxwYXo%2Brg%40mail.gmail.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/CAA%3DX%3DqeGwJ1OwOQw0C4jAr7nzWJUsKGeryAQ%3DgoicuJbCWr5HQ%40mail.gmail.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/a1e9a02f-b6bd-4f03-a2e5-808bbfbea80c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Query question

[Avraham Serour]

Personally I don't see obvious errors, but it is hard to tell with only
this small snippet.

If it works what are you worried about? Performance? If you are having
performance problems you can try caching the query

On Mon, Jul 13, 2015 at 6:08 PM, Joss Ingram  wrote:

> Ok, thanks fair enough.
>
> my Question then is :
>
> Is the query above bad?
>
> On Mon, Jul 13, 2015 at 4:00 PM, Avraham Serour  wrote:
>
>> You told a story but didn't ask a question
>>
>> On Mon, Jul 13, 2015 at 5:54 PM, Joss Ingram 
>> wrote:
>>
>>> I'm not getting much joy with my question, is there too little info?
>>>
>>> On Friday, 10 July 2015 15:31:50 UTC+1, Joss Ingram wrote:

 Hi,

 In my models I have an event index page type, which can have child
 event pages, each event page can be tagged (using taggit), and I want to
 produce a list of distinct tags used in the events that belong to that
 event index on the index itself. I'm using the Django CMS Wagtail, but I
 guess this is really a Django ORM query question. I''ve got some code as
 part of my model which is working but I don't think it's the most efficient
 way of doing this. Still trying unlearn doing things with SQL and learn how
 the same thing should be done in Django.

 My code as part of the event index class is

 @property
 def event_tag_list(self):
 child_events = EventPage.objects.live().descendant_of(self)
 return
 Tag.objects.filter(mysite_eventpagetag_items__isnull=False).filter(mysite_eventpagetag_items__content_object_id=child_events).order_by('slug').distinct('slug')

 Any feedback would be appreciated.

 Thanks

 Joss



  --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/61cf7c97-ae40-4c71-a094-ff1d819a7ce5%40googlegroups.com
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Django users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/django-users/qB8W-I3jJWg/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAFWa6t%2B1dpT%3D3oYG%2BqRopr9vWWXwY573%2BM%2BzkuT8ijxwYXo%2Brg%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAA%3DX%3DqeGwJ1OwOQw0C4jAr7nzWJUsKGeryAQ%3DgoicuJbCWr5HQ%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6tJKaa08QFDgsdRQdqaL-T4UT3XvLSFkh%2BAmSSP6VW774w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Query question

[Joss Ingram]

Ok, thanks fair enough.

my Question then is :

Is the query above bad?

On Mon, Jul 13, 2015 at 4:00 PM, Avraham Serour  wrote:

> You told a story but didn't ask a question
>
> On Mon, Jul 13, 2015 at 5:54 PM, Joss Ingram 
> wrote:
>
>> I'm not getting much joy with my question, is there too little info?
>>
>> On Friday, 10 July 2015 15:31:50 UTC+1, Joss Ingram wrote:
>>>
>>> Hi,
>>>
>>> In my models I have an event index page type, which can have child event
>>> pages, each event page can be tagged (using taggit), and I want to produce
>>> a list of distinct tags used in the events that belong to that event index
>>> on the index itself. I'm using the Django CMS Wagtail, but I guess this is
>>> really a Django ORM query question. I''ve got some code as part of my model
>>> which is working but I don't think it's the most efficient way of doing
>>> this. Still trying unlearn doing things with SQL and learn how the same
>>> thing should be done in Django.
>>>
>>> My code as part of the event index class is
>>>
>>> @property
>>> def event_tag_list(self):
>>> child_events = EventPage.objects.live().descendant_of(self)
>>> return
>>> Tag.objects.filter(mysite_eventpagetag_items__isnull=False).filter(mysite_eventpagetag_items__content_object_id=child_events).order_by('slug').distinct('slug')
>>>
>>> Any feedback would be appreciated.
>>>
>>> Thanks
>>>
>>> Joss
>>>
>>>
>>>
>>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/61cf7c97-ae40-4c71-a094-ff1d819a7ce5%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-users/qB8W-I3jJWg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAFWa6t%2B1dpT%3D3oYG%2BqRopr9vWWXwY573%2BM%2BzkuT8ijxwYXo%2Brg%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAA%3DX%3DqeGwJ1OwOQw0C4jAr7nzWJUsKGeryAQ%3DgoicuJbCWr5HQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Query question

[Avraham Serour]

You told a story but didn't ask a question

On Mon, Jul 13, 2015 at 5:54 PM, Joss Ingram  wrote:

> I'm not getting much joy with my question, is there too little info?
>
> On Friday, 10 July 2015 15:31:50 UTC+1, Joss Ingram wrote:
>>
>> Hi,
>>
>> In my models I have an event index page type, which can have child event
>> pages, each event page can be tagged (using taggit), and I want to produce
>> a list of distinct tags used in the events that belong to that event index
>> on the index itself. I'm using the Django CMS Wagtail, but I guess this is
>> really a Django ORM query question. I''ve got some code as part of my model
>> which is working but I don't think it's the most efficient way of doing
>> this. Still trying unlearn doing things with SQL and learn how the same
>> thing should be done in Django.
>>
>> My code as part of the event index class is
>>
>> @property
>> def event_tag_list(self):
>> child_events = EventPage.objects.live().descendant_of(self)
>> return
>> Tag.objects.filter(mysite_eventpagetag_items__isnull=False).filter(mysite_eventpagetag_items__content_object_id=child_events).order_by('slug').distinct('slug')
>>
>> Any feedback would be appreciated.
>>
>> Thanks
>>
>> Joss
>>
>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/61cf7c97-ae40-4c71-a094-ff1d819a7ce5%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6t%2B1dpT%3D3oYG%2BqRopr9vWWXwY573%2BM%2BzkuT8ijxwYXo%2Brg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Query question

[Joss Ingram]

I'm not getting much joy with my question, is there too little info? 

On Friday, 10 July 2015 15:31:50 UTC+1, Joss Ingram wrote:
>
> Hi, 
>
> In my models I have an event index page type, which can have child event 
> pages, each event page can be tagged (using taggit), and I want to produce 
> a list of distinct tags used in the events that belong to that event index 
> on the index itself. I'm using the Django CMS Wagtail, but I guess this is 
> really a Django ORM query question. I''ve got some code as part of my model 
> which is working but I don't think it's the most efficient way of doing 
> this. Still trying unlearn doing things with SQL and learn how the same 
> thing should be done in Django.
>
> My code as part of the event index class is 
>
> @property
> def event_tag_list(self):
> child_events = EventPage.objects.live().descendant_of(self)
> return 
> Tag.objects.filter(mysite_eventpagetag_items__isnull=False).filter(mysite_eventpagetag_items__content_object_id=child_events).order_by('slug').distinct('slug')
>
> Any feedback would be appreciated.
>
> Thanks
>
> Joss
>
> 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/61cf7c97-ae40-4c71-a094-ff1d819a7ce5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Michael P. Soulier]

On 04/09/14 Benjamin Scherrey said:

> The short answer to your question is no, the Django ORM is not inherently
> slower in that regard and it's very likely something you're doing. The

Given that it's basically

for obj in foo.objects.all():
obj.prop = new_value
obj.save()

I fail to see how it's something that I am doing. The whole thing runs in a
single transaction, and the only other addition would be Django's signals. I
believe I had none registered but I'll double check.

I should note that while sqlalchemy was 48 times faster, raw sql was roughly
100 times faster.

> Impossible to give you any more specific to your particular problem without
> seeing code, of course. That said, some common issues when grabbing
> individual model instances related to a larger query are often dramatically
> improved by using select_related() or fetch_related() as appropriate. Also

There is a foreign key involved in this model, not that it is being accessed,
but I can update the query and retry.

> Otherwise here's a decent little writeup of a good approach to providing
> better access to your ORM models:
> http://www.dabapps.com/blog/higher-level-query-api-django-orm/

I'll take a look, thanks.

Mike

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/20140904141000.GE11182%40digitaltorque.ca.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Michael P. Soulier]

On 03/09/14 Tom Lockhart said:

> I haven't had to deal with this myself, but the speed difference smacks of
> transactional issues. If you can run your loop by wrapping all of it or
> pieces of it (say, 100 or 1000 chunks) in a single transaction you'll
> probably see some significant speedup.

Yeah I tried that, and noticed no difference.

Mike

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/20140904140612.GD11182%40digitaltorque.ca.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Michael P. Soulier]

On 04/09/14 Tom Evans said:

> Is the update invariant? By using the ORM like this:

As I said, each update is unique and they cannot be batched.

> Are both Django and the sqlalchemy doing the same sort of update?

Yes. Identical.

Mike

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/20140904140538.GC11182%40digitaltorque.ca.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Tom Evans]

On Thu, Sep 4, 2014 at 12:22 AM, msoulier  wrote:
> Hi,
>
> I am looking at Django's performance with respect to modifying large numbers
> of objects, each in a unique way that cannot be batched. If I make a simple
> change to one of my Django models and save(), and then do the same thing in
> sqlalchemy, I notice a performance difference of about 48 times as far as
> the rate that the objects are processed to my postgresql db.
>
> The code is a simple property update and save, in a loop, trying to process
> as many objects as possible.

Is the update invariant? By using the ORM like this:

  for obj in MyObject.objects.all():
  obj.foo = 'hello'
  obj.save()

then you have to pull all the data for each object out of the
database, convert the raw DB column to it's python type, assign it to
a model instance, convert each python field back to its raw DB value
and save it in the database. That is N+1 queries, and many
conversions.

If the update is invariant, you can apply it without any of the
overhead, only 1 query and one conversion:

  MyObject.objects.all().update(foo='hello')

If the update doesn't depend on the other fields in the model, this
avoids some of the overhead, still N+1 queries but virtually no
conversion overhead:

  for pk in MyObject.objects.all().values_list('pk', flat=True):
  MyObject.objects.filter(pk=pk).update(foo=make_new_foo())

>
> Is the Django ORM known to be slower in this regard, or is it likely
> something that I'm doing?

Are both Django and the sqlalchemy doing the same sort of update?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFHbX1J6GGtV8MAJ8vA8QRyC8KVuwacULPvLSRiY%2BRd%2BMwcOng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Sithembewena Lloyd Dube]

Thanks to the OP for asking this, and to all who answered. Ben, special
stars to you - you just shared some very valuable insight into efficieint
use of the ORM (wasn't obvious to me).

Kind regards,
Lloyd


On Thu, Sep 4, 2014 at 2:21 AM, Benjamin Scherrey 
wrote:

> The short answer to your question is no, the Django ORM is not inherently
> slower in that regard and it's very likely something you're doing. The
> useful answer is probably more complicated. :-) Naive usage of the ORM
> without an understanding of how it translates to SQL is likely to result in
> some really awful non-performant database requests for any reasonably
> complex models/queries. The good news is that it isn't very hard to get
> quite good performance out of it.
>
> Impossible to give you any more specific to your particular problem
> without seeing code, of course. That said, some common issues when grabbing
> individual model instances related to a larger query are often dramatically
> improved by using select_related() or fetch_related() as appropriate. Also
> for doing large amounts of writes/updates you should look into doing the
> transaction management yourself as Tom suggests. Postgres can really fly
> once you understand how the ORM works with it. MySQL does nicely as well
> but I have less experience with it. Ultimately, if your request can't be
> efficiently modeled with the ORM (rare but does happen) then you can use
> .extra() to pass in some direct SQL quite easily.
>
> Otherwise here's a decent little writeup of a good approach to providing
> better access to your ORM models:
> http://www.dabapps.com/blog/higher-level-query-api-django-orm/
>
> Good luck,
>
>   -- Ben
>
>
> On Thu, Sep 4, 2014 at 6:22 AM, msoulier 
> wrote:
>
>> Hi,
>>
>> I am looking at Django's performance with respect to modifying large
>> numbers of objects, each in a unique way that cannot be batched. If I make
>> a simple change to one of my Django models and save(), and then do the same
>> thing in sqlalchemy, I notice a performance difference of about 48 times as
>> far as the rate that the objects are processed to my postgresql db.
>>
>> The code is a simple property update and save, in a loop, trying to
>> process as many objects as possible.
>>
>> Is the Django ORM known to be slower in this regard, or is it likely
>> something that I'm doing?
>>
>> Thanks,
>> Mike
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/9049bff2-470e-4560-b93f-dee56bc924d4%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Chief Systems Architect Proteus Technologies 
> Chief Fan Biggest Fan Productions 
> Personal blog where I am not your demographic
> .
>
> This email intended solely for those who have received it. If you have
> received this email by accident - well lucky you!!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAHN%3D9D6ATHXVzDhwoXBx1xG0hS9n%3Do6egG450Js1Hrwv9KQTBg%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Regards,
Sithu Lloyd Dube

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAH-SnCCXzJYbdURYc2a-6ViouO8ZN4A-5FjJ%3Dk%3DSp2hNzgvNdQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Benjamin Scherrey]

The short answer to your question is no, the Django ORM is not inherently
slower in that regard and it's very likely something you're doing. The
useful answer is probably more complicated. :-) Naive usage of the ORM
without an understanding of how it translates to SQL is likely to result in
some really awful non-performant database requests for any reasonably
complex models/queries. The good news is that it isn't very hard to get
quite good performance out of it.

Impossible to give you any more specific to your particular problem without
seeing code, of course. That said, some common issues when grabbing
individual model instances related to a larger query are often dramatically
improved by using select_related() or fetch_related() as appropriate. Also
for doing large amounts of writes/updates you should look into doing the
transaction management yourself as Tom suggests. Postgres can really fly
once you understand how the ORM works with it. MySQL does nicely as well
but I have less experience with it. Ultimately, if your request can't be
efficiently modeled with the ORM (rare but does happen) then you can use
.extra() to pass in some direct SQL quite easily.

Otherwise here's a decent little writeup of a good approach to providing
better access to your ORM models:
http://www.dabapps.com/blog/higher-level-query-api-django-orm/

Good luck,

  -- Ben


On Thu, Sep 4, 2014 at 6:22 AM, msoulier  wrote:

> Hi,
>
> I am looking at Django's performance with respect to modifying large
> numbers of objects, each in a unique way that cannot be batched. If I make
> a simple change to one of my Django models and save(), and then do the same
> thing in sqlalchemy, I notice a performance difference of about 48 times as
> far as the rate that the objects are processed to my postgresql db.
>
> The code is a simple property update and save, in a loop, trying to
> process as many objects as possible.
>
> Is the Django ORM known to be slower in this regard, or is it likely
> something that I'm doing?
>
> Thanks,
> Mike
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/9049bff2-470e-4560-b93f-dee56bc924d4%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Chief Systems Architect Proteus Technologies 
Chief Fan Biggest Fan Productions 
Personal blog where I am not your demographic
.

This email intended solely for those who have received it. If you have
received this email by accident - well lucky you!!

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAHN%3D9D6ATHXVzDhwoXBx1xG0hS9n%3Do6egG450Js1Hrwv9KQTBg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM performance

[Tom Lockhart]

On 2014-09-03, at 4:22 PM, msoulier  wrote:

> Hi,
> 
> I am looking at Django's performance with respect to modifying large numbers 
> of objects, each in a unique way that cannot be batched. If I make a simple 
> change to one of my Django models and save(), and then do the same thing in 
> sqlalchemy, I notice a performance difference of about 48 times as far as the 
> rate that the objects are processed to my postgresql db.
> 
> The code is a simple property update and save, in a loop, trying to process 
> as many objects as possible.
> 
> Is the Django ORM known to be slower in this regard, or is it likely 
> something that I'm doing?

I haven't had to deal with this myself, but the speed difference smacks of 
transactional issues. If you can run your loop by wrapping all of it or pieces 
of it (say, 100 or 1000 chunks) in a single transaction you'll probably see 
some significant speedup.

https://docs.djangoproject.com/en/dev/topics/db/transactions/

hth

- Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/1193F2BD-2597-4D68-A8EE-D593FFD084EF%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Question

[Bill Freeman]

>From the lack of other folks jumping in saying "do it this way", I'm going
to guess that this is a hard problem.  I am not a database heavyweight.
(IANADbH)  I especially don't know about limitations of MySQL.

It seems that you need to calculate a maximum size of Llama objects, but
separately for each LlamaHerd, and then you have to select, again for each
LlamaHerd, choose a Llama of hte corresponding size (what if there are
multiple animals in that herd with that size?) to include in the result,
and then attach an order by date of birth to the final result.

I don't know how to do that in one SQL statement.  I do know from
experience that it's easy, by choosing a naive approach, to have such an
aggregate size of join tables that the database starts to swap.

So lets hope that a DbG (Database Guru) pipes up with an answer.


On Wed, Apr 2, 2014 at 9:31 PM, Justin Holmes wrote:

> I think you misunderstand: I am not actually tracking llama herds. :-)
>
> This is just example code; I haven't actually tested it in the way that
> you suggest.  The problem isn't with this code in particular, it's with
> this concept in general.  This is a question I have had again and again in
> the past few years - this code is just a way to articulate it.
>
> So, in short - no, the code won't work as now written, and it's obvious
> why: annotate doesn't assign instances, it assigns results.
>
> I don't think I need an annotation expert per se; it may well be that
> annotate is the wrong way to do this.
>
> My SQL isn't particularly strong, but if I were writing this in SQL, I
> suspect that GROUP_BY is necessary or at least plausible.  I know that
> Django doesn't directly expose GROUP_BY, but it looks likes
> QuerySet.values() uses it; so I wonder if that fits into the picture
> somehow?
>
>
> On Wed, Apr 2, 2014 at 12:35 PM, Bill Freeman  wrote:
>
>> My point was that annotate might work after the other problems were
>> fixed.  Have you re-tested since?  If your annotate still doesn't work then
>> perhaps someone with more annotate experience than I will comment.
>>
>>
>>
>> On Wed, Apr 2, 2014 at 12:04 PM, Justin Holmes 
>> wrote:
>>
>>> No no, the annotate doesn't work - that's the whole reason for my
>>> question.  See, the annotate doesn't associate instances with the annotated
>>> attribute, it associated results.
>>>
>>> (BTW, good looking out - I have once again fixed the SO question :-))
>>>
>>>
>>> On Wed, Apr 2, 2014 at 10:17 AM, Bill Freeman  wrote:
>>>
 1.  You now have a foreign key to Herd, but you don't have a model by
 that name.  It's called LlamaHerd.

 2.  I was misinterpreting what you wanted (couldn't get past the lack
 of a relationship while reading0.  Once the relationship is fixed, your
 original annotate might work.


 On Tue, Apr 1, 2014 at 6:20 PM, Justin Holmes 
 wrote:

> You say "use the reverse relation manager in LlamaHerd to build up
> your annotation," but that answer the question.
>
> How can I sort LlamaHerd objects by the DOB of their largest llama?
>
>
> On Tue, Apr 1, 2014 at 6:13 PM, Justin Holmes  > wrote:
>
>> Yes, absolutely - my bad.  I have updated to SO question.  I meant
>> the models to look like:
>>
>> class LlamaHerd(models.Model):
>> shepherd = ForeignKey(User)
>>
>> class Llama(models.Model):
>> size = FloatField()
>> dob =
>>  FloatField
>> herd = ForeignKey(Herd, related_name="llamas")
>>
>>
>>
>>
>> On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman wrote:
>>
>>> Aren't you missing a ForeignKey relationship to tell which LlamaHerd
>>> a Llama belongs to?
>>>
>>> Then you would use the reverse relation manager in LlamaHerd to
>>> build up your annotation.
>>>
>>>
>>> On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes <
>>> jus...@justinholmes.com> wrote:
>>>
 I have come across this problem in several Django projects, but
 only just now worked it out in my head enough to make it a cognizable
 question.

 I have also posted it on StackOverflow, here:
 http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item


 Consider this example:

 class LlamaHerd(models.Model):
 shepherd = ForeignKey(User)

 class Llama(models.Model):
 size = FloatField()
 dob = FloatField

 How can I now make a query to get LlamaHerd objects, sorted by the
 date of birth of their largest llama?

 It almost feels like I might be able to use annotate:

 LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')

 ...but the annotate here creates fields with float values, not with
 the model instances.

>

Re: ORM Question

[Justin Holmes]

I think you misunderstand: I am not actually tracking llama herds. :-)

This is just example code; I haven't actually tested it in the way that you
suggest.  The problem isn't with this code in particular, it's with this
concept in general.  This is a question I have had again and again in the
past few years - this code is just a way to articulate it.

So, in short - no, the code won't work as now written, and it's obvious
why: annotate doesn't assign instances, it assigns results.

I don't think I need an annotation expert per se; it may well be that
annotate is the wrong way to do this.

My SQL isn't particularly strong, but if I were writing this in SQL, I
suspect that GROUP_BY is necessary or at least plausible.  I know that
Django doesn't directly expose GROUP_BY, but it looks likes
QuerySet.values() uses it; so I wonder if that fits into the picture
somehow?


On Wed, Apr 2, 2014 at 12:35 PM, Bill Freeman  wrote:

> My point was that annotate might work after the other problems were
> fixed.  Have you re-tested since?  If your annotate still doesn't work then
> perhaps someone with more annotate experience than I will comment.
>
>
>
> On Wed, Apr 2, 2014 at 12:04 PM, Justin Holmes wrote:
>
>> No no, the annotate doesn't work - that's the whole reason for my
>> question.  See, the annotate doesn't associate instances with the annotated
>> attribute, it associated results.
>>
>> (BTW, good looking out - I have once again fixed the SO question :-))
>>
>>
>> On Wed, Apr 2, 2014 at 10:17 AM, Bill Freeman  wrote:
>>
>>> 1.  You now have a foreign key to Herd, but you don't have a model by
>>> that name.  It's called LlamaHerd.
>>>
>>> 2.  I was misinterpreting what you wanted (couldn't get past the lack of
>>> a relationship while reading0.  Once the relationship is fixed, your
>>> original annotate might work.
>>>
>>>
>>> On Tue, Apr 1, 2014 at 6:20 PM, Justin Holmes 
>>> wrote:
>>>
 You say "use the reverse relation manager in LlamaHerd to build up your
 annotation," but that answer the question.

 How can I sort LlamaHerd objects by the DOB of their largest llama?


 On Tue, Apr 1, 2014 at 6:13 PM, Justin Holmes 
 wrote:

> Yes, absolutely - my bad.  I have updated to SO question.  I meant the
> models to look like:
>
> class LlamaHerd(models.Model):
> shepherd = ForeignKey(User)
>
> class Llama(models.Model):
> size = FloatField()
> dob =
>  FloatField
> herd = ForeignKey(Herd, related_name="llamas")
>
>
>
>
> On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman wrote:
>
>> Aren't you missing a ForeignKey relationship to tell which LlamaHerd
>> a Llama belongs to?
>>
>> Then you would use the reverse relation manager in LlamaHerd to build
>> up your annotation.
>>
>>
>> On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes <
>> jus...@justinholmes.com> wrote:
>>
>>> I have come across this problem in several Django projects, but only
>>> just now worked it out in my head enough to make it a cognizable 
>>> question.
>>>
>>> I have also posted it on StackOverflow, here:
>>> http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item
>>>
>>>
>>> Consider this example:
>>>
>>> class LlamaHerd(models.Model):
>>> shepherd = ForeignKey(User)
>>>
>>> class Llama(models.Model):
>>> size = FloatField()
>>> dob = FloatField
>>>
>>> How can I now make a query to get LlamaHerd objects, sorted by the
>>> date of birth of their largest llama?
>>>
>>> It almost feels like I might be able to use annotate:
>>>
>>> LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')
>>>
>>> ...but the annotate here creates fields with float values, not with
>>> the model instances.
>>>
>>>
>>> --
>>> Justin Holmes
>>> Chief Chocobo Breeder, slashRoot
>>>
>>> slashRoot: Coffee House and Tech Dojo
>>> New Paltz, NY 12561
>>> 845.633.8330
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

Re: ORM Question

[Bill Freeman]

My point was that annotate might work after the other problems were fixed.
Have you re-tested since?  If your annotate still doesn't work then perhaps
someone with more annotate experience than I will comment.



On Wed, Apr 2, 2014 at 12:04 PM, Justin Holmes wrote:

> No no, the annotate doesn't work - that's the whole reason for my
> question.  See, the annotate doesn't associate instances with the annotated
> attribute, it associated results.
>
> (BTW, good looking out - I have once again fixed the SO question :-))
>
>
> On Wed, Apr 2, 2014 at 10:17 AM, Bill Freeman  wrote:
>
>> 1.  You now have a foreign key to Herd, but you don't have a model by
>> that name.  It's called LlamaHerd.
>>
>> 2.  I was misinterpreting what you wanted (couldn't get past the lack of
>> a relationship while reading0.  Once the relationship is fixed, your
>> original annotate might work.
>>
>>
>> On Tue, Apr 1, 2014 at 6:20 PM, Justin Holmes wrote:
>>
>>> You say "use the reverse relation manager in LlamaHerd to build up your
>>> annotation," but that answer the question.
>>>
>>> How can I sort LlamaHerd objects by the DOB of their largest llama?
>>>
>>>
>>> On Tue, Apr 1, 2014 at 6:13 PM, Justin Holmes 
>>> wrote:
>>>
 Yes, absolutely - my bad.  I have updated to SO question.  I meant the
 models to look like:

 class LlamaHerd(models.Model):
 shepherd = ForeignKey(User)

 class Llama(models.Model):
 size = FloatField()
 dob =
  FloatField
 herd = ForeignKey(Herd, related_name="llamas")




 On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman  wrote:

> Aren't you missing a ForeignKey relationship to tell which LlamaHerd a
> Llama belongs to?
>
> Then you would use the reverse relation manager in LlamaHerd to build
> up your annotation.
>
>
> On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes  > wrote:
>
>> I have come across this problem in several Django projects, but only
>> just now worked it out in my head enough to make it a cognizable 
>> question.
>>
>> I have also posted it on StackOverflow, here:
>> http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item
>>
>>
>> Consider this example:
>>
>> class LlamaHerd(models.Model):
>> shepherd = ForeignKey(User)
>>
>> class Llama(models.Model):
>> size = FloatField()
>> dob = FloatField
>>
>> How can I now make a query to get LlamaHerd objects, sorted by the
>> date of birth of their largest llama?
>>
>> It almost feels like I might be able to use annotate:
>>
>> LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')
>>
>> ...but the annotate here creates fields with float values, not with
>> the model instances.
>>
>>
>> --
>> Justin Holmes
>> Chief Chocobo Breeder, slashRoot
>>
>> slashRoot: Coffee House and Tech Dojo
>> New Paltz, NY 12561
>> 845.633.8330
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAB%2BAj0srKNzRs%3DzbsPm%2BobxLNFCcjRccGSVeYykF1PSnJc-ydQ%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



 --
 Justin Holmes
 Chief Chocobo Breeder, slashRoot

 slashRoot: Coffee House and Tech Dojo
 New Paltz, NY 12561
 845.633.8330

>>>
>>>
>>>
>>> --
>>> Justin Holmes
>>> Chief Chocobo Breeder, slashRoot
>>>
>>> slashRoot

Re: ORM Question

[Justin Holmes]

No no, the annotate doesn't work - that's the whole reason for my
question.  See, the annotate doesn't associate instances with the annotated
attribute, it associated results.

(BTW, good looking out - I have once again fixed the SO question :-))


On Wed, Apr 2, 2014 at 10:17 AM, Bill Freeman  wrote:

> 1.  You now have a foreign key to Herd, but you don't have a model by that
> name.  It's called LlamaHerd.
>
> 2.  I was misinterpreting what you wanted (couldn't get past the lack of a
> relationship while reading0.  Once the relationship is fixed, your original
> annotate might work.
>
>
> On Tue, Apr 1, 2014 at 6:20 PM, Justin Holmes wrote:
>
>> You say "use the reverse relation manager in LlamaHerd to build up your
>> annotation," but that answer the question.
>>
>> How can I sort LlamaHerd objects by the DOB of their largest llama?
>>
>>
>> On Tue, Apr 1, 2014 at 6:13 PM, Justin Holmes wrote:
>>
>>> Yes, absolutely - my bad.  I have updated to SO question.  I meant the
>>> models to look like:
>>>
>>> class LlamaHerd(models.Model):
>>> shepherd = ForeignKey(User)
>>>
>>> class Llama(models.Model):
>>> size = FloatField()
>>> dob =
>>>  FloatField
>>> herd = ForeignKey(Herd, related_name="llamas")
>>>
>>>
>>>
>>>
>>> On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman  wrote:
>>>
 Aren't you missing a ForeignKey relationship to tell which LlamaHerd a
 Llama belongs to?

 Then you would use the reverse relation manager in LlamaHerd to build
 up your annotation.


 On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes 
 wrote:

> I have come across this problem in several Django projects, but only
> just now worked it out in my head enough to make it a cognizable question.
>
> I have also posted it on StackOverflow, here:
> http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item
>
>
> Consider this example:
>
> class LlamaHerd(models.Model):
> shepherd = ForeignKey(User)
>
> class Llama(models.Model):
> size = FloatField()
> dob = FloatField
>
> How can I now make a query to get LlamaHerd objects, sorted by the
> date of birth of their largest llama?
>
> It almost feels like I might be able to use annotate:
>
> LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')
>
> ...but the annotate here creates fields with float values, not with
> the model instances.
>
>
> --
> Justin Holmes
> Chief Chocobo Breeder, slashRoot
>
> slashRoot: Coffee House and Tech Dojo
> New Paltz, NY 12561
> 845.633.8330
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

  --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users+unsubscr...@googlegroups.com.
 To post to this group, send email to django-users@googlegroups.com.
 Visit this group at http://groups.google.com/group/django-users.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-users/CAB%2BAj0srKNzRs%3DzbsPm%2BobxLNFCcjRccGSVeYykF1PSnJc-ydQ%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> --
>>> Justin Holmes
>>> Chief Chocobo Breeder, slashRoot
>>>
>>> slashRoot: Coffee House and Tech Dojo
>>> New Paltz, NY 12561
>>> 845.633.8330
>>>
>>
>>
>>
>> --
>> Justin Holmes
>> Chief Chocobo Breeder, slashRoot
>>
>> slashRoot: Coffee House and Tech Dojo
>> New Paltz, NY 12561
>> 845.633.8330
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.

Re: ORM Question

[Bill Freeman]

1.  You now have a foreign key to Herd, but you don't have a model by that
name.  It's called LlamaHerd.

2.  I was misinterpreting what you wanted (couldn't get past the lack of a
relationship while reading0.  Once the relationship is fixed, your original
annotate might work.


On Tue, Apr 1, 2014 at 6:20 PM, Justin Holmes wrote:

> You say "use the reverse relation manager in LlamaHerd to build up your
> annotation," but that answer the question.
>
> How can I sort LlamaHerd objects by the DOB of their largest llama?
>
>
> On Tue, Apr 1, 2014 at 6:13 PM, Justin Holmes wrote:
>
>> Yes, absolutely - my bad.  I have updated to SO question.  I meant the
>> models to look like:
>>
>> class LlamaHerd(models.Model):
>> shepherd = ForeignKey(User)
>>
>> class Llama(models.Model):
>> size = FloatField()
>> dob =
>>  FloatField
>> herd = ForeignKey(Herd, related_name="llamas")
>>
>>
>>
>>
>> On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman  wrote:
>>
>>> Aren't you missing a ForeignKey relationship to tell which LlamaHerd a
>>> Llama belongs to?
>>>
>>> Then you would use the reverse relation manager in LlamaHerd to build up
>>> your annotation.
>>>
>>>
>>> On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes 
>>> wrote:
>>>
 I have come across this problem in several Django projects, but only
 just now worked it out in my head enough to make it a cognizable question.

 I have also posted it on StackOverflow, here:
 http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item


 Consider this example:

 class LlamaHerd(models.Model):
 shepherd = ForeignKey(User)

 class Llama(models.Model):
 size = FloatField()
 dob = FloatField

 How can I now make a query to get LlamaHerd objects, sorted by the date
 of birth of their largest llama?

 It almost feels like I might be able to use annotate:

 LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')

 ...but the annotate here creates fields with float values, not with the
 model instances.


 --
 Justin Holmes
 Chief Chocobo Breeder, slashRoot

 slashRoot: Coffee House and Tech Dojo
 New Paltz, NY 12561
 845.633.8330

 --
 You received this message because you are subscribed to the Google
 Groups "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-users+unsubscr...@googlegroups.com.
 To post to this group, send email to django-users@googlegroups.com.
 Visit this group at http://groups.google.com/group/django-users.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAB%2BAj0srKNzRs%3DzbsPm%2BobxLNFCcjRccGSVeYykF1PSnJc-ydQ%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Justin Holmes
>> Chief Chocobo Breeder, slashRoot
>>
>> slashRoot: Coffee House and Tech Dojo
>> New Paltz, NY 12561
>> 845.633.8330
>>
>
>
>
> --
> Justin Holmes
> Chief Chocobo Breeder, slashRoot
>
> slashRoot: Coffee House and Tech Dojo
> New Paltz, NY 12561
> 845.633.8330
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAMGywB4V3zpbqr7v1kcvOi8%2BoqLPm0nr18dGStaedys494%2BOag%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this

Re: ORM Question

[Justin Holmes]

You say "use the reverse relation manager in LlamaHerd to build up your
annotation," but that answer the question.

How can I sort LlamaHerd objects by the DOB of their largest llama?


On Tue, Apr 1, 2014 at 6:13 PM, Justin Holmes wrote:

> Yes, absolutely - my bad.  I have updated to SO question.  I meant the
> models to look like:
>
> class LlamaHerd(models.Model):
> shepherd = ForeignKey(User)
>
> class Llama(models.Model):
> size = FloatField()
> dob =
>  FloatField
> herd = ForeignKey(Herd, related_name="llamas")
>
>
>
>
> On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman  wrote:
>
>> Aren't you missing a ForeignKey relationship to tell which LlamaHerd a
>> Llama belongs to?
>>
>> Then you would use the reverse relation manager in LlamaHerd to build up
>> your annotation.
>>
>>
>> On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes wrote:
>>
>>> I have come across this problem in several Django projects, but only
>>> just now worked it out in my head enough to make it a cognizable question.
>>>
>>> I have also posted it on StackOverflow, here:
>>> http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item
>>>
>>>
>>> Consider this example:
>>>
>>> class LlamaHerd(models.Model):
>>> shepherd = ForeignKey(User)
>>>
>>> class Llama(models.Model):
>>> size = FloatField()
>>> dob = FloatField
>>>
>>> How can I now make a query to get LlamaHerd objects, sorted by the date
>>> of birth of their largest llama?
>>>
>>> It almost feels like I might be able to use annotate:
>>>
>>> LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')
>>>
>>> ...but the annotate here creates fields with float values, not with the
>>> model instances.
>>>
>>>
>>> --
>>> Justin Holmes
>>> Chief Chocobo Breeder, slashRoot
>>>
>>> slashRoot: Coffee House and Tech Dojo
>>> New Paltz, NY 12561
>>> 845.633.8330
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-users@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-users.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAB%2BAj0srKNzRs%3DzbsPm%2BobxLNFCcjRccGSVeYykF1PSnJc-ydQ%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Justin Holmes
> Chief Chocobo Breeder, slashRoot
>
> slashRoot: Coffee House and Tech Dojo
> New Paltz, NY 12561
> 845.633.8330
>



-- 
Justin Holmes
Chief Chocobo Breeder, slashRoot

slashRoot: Coffee House and Tech Dojo
New Paltz, NY 12561
845.633.8330

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAMGywB4V3zpbqr7v1kcvOi8%2BoqLPm0nr18dGStaedys494%2BOag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Question

[Justin Holmes]

Yes, absolutely - my bad.  I have updated to SO question.  I meant the
models to look like:

class LlamaHerd(models.Model):
shepherd = ForeignKey(User)

class Llama(models.Model):
size = FloatField()
dob = FloatField
herd = ForeignKey(Herd, related_name="llamas")




On Tue, Apr 1, 2014 at 6:09 PM, Bill Freeman  wrote:

> Aren't you missing a ForeignKey relationship to tell which LlamaHerd a
> Llama belongs to?
>
> Then you would use the reverse relation manager in LlamaHerd to build up
> your annotation.
>
>
> On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes wrote:
>
>> I have come across this problem in several Django projects, but only just
>> now worked it out in my head enough to make it a cognizable question.
>>
>> I have also posted it on StackOverflow, here:
>> http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item
>>
>>
>> Consider this example:
>>
>> class LlamaHerd(models.Model):
>> shepherd = ForeignKey(User)
>>
>> class Llama(models.Model):
>> size = FloatField()
>> dob = FloatField
>>
>> How can I now make a query to get LlamaHerd objects, sorted by the date
>> of birth of their largest llama?
>>
>> It almost feels like I might be able to use annotate:
>>
>> LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')
>>
>> ...but the annotate here creates fields with float values, not with the
>> model instances.
>>
>>
>> --
>> Justin Holmes
>> Chief Chocobo Breeder, slashRoot
>>
>> slashRoot: Coffee House and Tech Dojo
>> New Paltz, NY 12561
>> 845.633.8330
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAB%2BAj0srKNzRs%3DzbsPm%2BobxLNFCcjRccGSVeYykF1PSnJc-ydQ%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Justin Holmes
Chief Chocobo Breeder, slashRoot

slashRoot: Coffee House and Tech Dojo
New Paltz, NY 12561
845.633.8330

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAMGywB6r1wu0hsTVH8JdpBMSWDLBWsH87%2B1H_vZ%2BDqr0Eu%2BSXw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Question

[Bill Freeman]

Aren't you missing a ForeignKey relationship to tell which LlamaHerd a
Llama belongs to?

Then you would use the reverse relation manager in LlamaHerd to build up
your annotation.


On Tue, Apr 1, 2014 at 5:52 PM, Justin Holmes wrote:

> I have come across this problem in several Django projects, but only just
> now worked it out in my head enough to make it a cognizable question.
>
> I have also posted it on StackOverflow, here:
> http://stackoverflow.com/questions/22797497/using-the-django-orm-to-order-by-a-value-in-a-distinct-related-item
>
>
> Consider this example:
>
> class LlamaHerd(models.Model):
> shepherd = ForeignKey(User)
>
> class Llama(models.Model):
> size = FloatField()
> dob = FloatField
>
> How can I now make a query to get LlamaHerd objects, sorted by the date of
> birth of their largest llama?
>
> It almost feels like I might be able to use annotate:
>
> LlamaHerd.annotate(largest_llama=Max('llama__size')).order_by('largest_llama__dob')
>
> ...but the annotate here creates fields with float values, not with the
> model instances.
>
>
> --
> Justin Holmes
> Chief Chocobo Breeder, slashRoot
>
> slashRoot: Coffee House and Tech Dojo
> New Paltz, NY 12561
> 845.633.8330
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAMGywB6gkBvhFs%2BP2PrtDah24VDCko8QK-1UCnSGugz_ts1v8g%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAB%2BAj0srKNzRs%3DzbsPm%2BobxLNFCcjRccGSVeYykF1PSnJc-ydQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ORM Prefetch related and only()

[Àlex Pérez]

As you say the raw sql always is the solution.

But I take a few time to think how that could be possible with the ORM, and
like
akaariai says the prefetch_related get the values for de _default_manager.

But if i defined a temporally manager that makes the only clausure it
works, but i don't know if is thread safe.



class OtherManager(models.Manager):
def get_query_set(self):
return
super(OtherManager,self).get_query_set().only("nombre")

class Especialidad(models.Model):
nombre = models.CharField(max_length=255)

objects = models.Manager()
otro = OtherManager()


Especialidad._default_manager = Especialidad.otro
tpl_vars["list"] = list(tpl_vars["list"])
Especialidad._default_manager = Especialidad.objects

Thank's!


2013/5/29 Christophe Pettus 

>
> On May 29, 2013, at 12:44 PM, Àlex Pérez wrote:
>
> > You know the way to simulate  the behaviour that i want.?
>
> Raw SQL.  At the point you are doing a multi-join query and selecting a
> subset of the fields to be returned, you probably should switch to raw SQL
> anyway.
>
> --
> -- Christophe Pettus
>x...@thebuild.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>


-- 
Alex Perez
alex.pe...@bebabum.com

 *bebabum* be successful

c/ Còrsega 301-303, Àtic 2
08008 Barcelona
http://www.bebabum.com
http://www.facebook.com/bebabum
http://twitter.com/bebabum

This message is intended exclusively for its addressee and may contain
information that is confidential and protected by professional privilege.
If you are not the intended recipient you are hereby notified that any
dissemination, copy or disclosure of this communication is strictly
prohibited by law.

Este mensaje se dirige exclusivamente a su destinatario y puede contener
información privilegiada o confidencial. Si no es vd. el destinatario
indicado,
queda notificado que la utilización, divulgación y/o copia sin autorización
está prohibida en virtud de la legislación vigente.

Le informamos que los datos personales que facilite/ha facilitado pasarán a
formar parte de un fichero responsabilidad de bebabum, S.L. y que tiene
por finalidad gestionar las relaciones con usted.
Tiene derecho al acceso, rectificación cancelación y oposición en nuestra
oficina ubicada en c/ Còrsega 301-303, Àtic 2 de Barcelona o a la dirección
de e-mail l...@bebabum.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ORM Prefetch related and only()

[Christophe Pettus]

On May 29, 2013, at 12:44 PM, Àlex Pérez wrote:

> You know the way to simulate  the behaviour that i want.?

Raw SQL.  At the point you are doing a multi-join query and selecting a subset 
of the fields to be returned, you probably should switch to raw SQL anyway.

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ORM Prefetch related and only()

[Àlex Pérez]

Ok thank's!!!

but you thinks thats the normal behaviour? this could be included in future
versions of the orm?

You know the way to simulate  the behaviour that i want.?

2013/5/29 akaariai 

> There is no way to alter the behavior of prefetch_related. It fetches
> the related objects with the query defined by the related manager and
> that's it.
>
>  - Anssi
>




-- 
Alex Perez
alex.pe...@bebabum.com

 *bebabum* be successful

c/ Còrsega 301-303, Àtic 2
08008 Barcelona
http://www.bebabum.com
http://www.facebook.com/bebabum
http://twitter.com/bebabum

This message is intended exclusively for its addressee and may contain
information that is confidential and protected by professional privilege.
If you are not the intended recipient you are hereby notified that any
dissemination, copy or disclosure of this communication is strictly
prohibited by law.

Este mensaje se dirige exclusivamente a su destinatario y puede contener
información privilegiada o confidencial. Si no es vd. el destinatario
indicado,
queda notificado que la utilización, divulgación y/o copia sin autorización
está prohibida en virtud de la legislación vigente.

Le informamos que los datos personales que facilite/ha facilitado pasarán a
formar parte de un fichero responsabilidad de bebabum, S.L. y que tiene
por finalidad gestionar las relaciones con usted.
Tiene derecho al acceso, rectificación cancelación y oposición en nuestra
oficina ubicada en c/ Còrsega 301-303, Àtic 2 de Barcelona o a la dirección
de e-mail l...@bebabum.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ORM Prefetch related and only()

[akaariai]

There is no way to alter the behavior of prefetch_related. It fetches
the related objects with the query defined by the related manager and
that's it.

 - Anssi

On 29 touko, 19:40, Àlex Pérez  wrote:
> I want to know if there is a way that the query that is executed to get the
> prefetch related information can get only certains values, actually that
> only works for select_related,
>
> May be i have done something wrong but the prefetch_related query is
> getting all the fields of the model.
>
> Someone can help me please?
>
> --
> Alex Perez
> alex.pe...@bebabum.com
>
>  *bebabum* be successful
>
> c/ Còrsega 301-303, Àtic 2
> 08008 
> Barcelonahttp://www.bebabum.comhttp://www.facebook.com/bebabumhttp://twitter.com/bebabum
>
> This message is intended exclusively for its addressee and may contain
> information that is confidential and protected by professional privilege.
> If you are not the intended recipient you are hereby notified that any
> dissemination, copy or disclosure of this communication is strictly
> prohibited by law.
>
> Este mensaje se dirige exclusivamente a su destinatario y puede contener
> información privilegiada o confidencial. Si no es vd. el destinatario
> indicado,
> queda notificado que la utilización, divulgación y/o copia sin autorización
> está prohibida en virtud de la legislación vigente.
>
> Le informamos que los datos personales que facilite/ha facilitado pasarán a
> formar parte de un fichero responsabilidad de bebabum, S.L. y que tiene
> por finalidad gestionar las relaciones con usted.
> Tiene derecho al acceso, rectificación cancelación y oposición en nuestra
> oficina ubicada en c/ Còrsega 301-303, Àtic 2 de Barcelona o a la dirección
> de e-mail l...@bebabum.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ORM, Oracle and UTF-8 encoding problem.

[Jani Tiainen]

10.1.2013 8:59, Ian Kelly kirjoitti:

On Wed, Jan 9, 2013 at 11:40 PM, Jani Tiainen  wrote:

If we just force using force_unicode everything works except in older
versions of cx_Oracle (our server had 5.0.4 or something) connection strings
can't be unicode for some reason.


Sure, that's why the check exists in the first place.  Prior to 5.1
cx_Oracle could be built either with Unicode or without.  If the
former, it would accept only unicode strings and would raise an
exception on byte strings.  If the latter, it would be exactly the
opposite.

Does it work for you using force_bytes with 5.0.4?



That's on my production server that runs 1.3.x version. smart_str (which 
detection selects) does not work.


using force_unicode works (except for connection string).

Also depending on what OCI client 10.2.0.5 or instant client 11.2 is 
used when compiling cx_Oracle causes variation. 10.2.0.5 doesn't work 
with smart_str while 11.2 does work.


Both can take plain unicode (u'') when using 
just cx_Oracle commands without any problems.


Note:

If I add manually some unicode to database Django can read it without 
any problems.


--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Ian Kelly]

On Wed, Jan 9, 2013 at 11:40 PM, Jani Tiainen  wrote:
> If we just force using force_unicode everything works except in older
> versions of cx_Oracle (our server had 5.0.4 or something) connection strings
> can't be unicode for some reason.

Sure, that's why the check exists in the first place.  Prior to 5.1
cx_Oracle could be built either with Unicode or without.  If the
former, it would accept only unicode strings and would raise an
exception on byte strings.  If the latter, it would be exactly the
opposite.

Does it work for you using force_bytes with 5.0.4?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Jani Tiainen]

9.1.2013 19:21, Ian Kelly kirjoitti:

On Wed, Jan 9, 2013 at 3:55 AM, Jani Tiainen  wrote:

Server is running Oracle Database 10g Release 10.2.0.5.0 - 64bit Production.
(EE edition)

and charset info:
NLS_CHARACTERSETWE8ISO8859P1
NLS_NCHAR_CHARACTERSET  AL16UTF16


Sorry, I meant your web server setup.



Windows 7, development server.

Staging server Ubuntu something (propably 10.04 LTS) 64bit

And symptoms were consistent. For some reason Django does something bad 
when it uses smart_str (and whatever that is in 1.5).


If we just force using force_unicode everything works except in older 
versions of cx_Oracle (our server had 5.0.4 or something) connection 
strings can't be unicode for some reason.


--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Ian Kelly]

On Wed, Jan 9, 2013 at 3:55 AM, Jani Tiainen  wrote:
> Server is running Oracle Database 10g Release 10.2.0.5.0 - 64bit Production.
> (EE edition)
>
> and charset info:
> NLS_CHARACTERSETWE8ISO8859P1
> NLS_NCHAR_CHARACTERSET  AL16UTF16

Sorry, I meant your web server setup.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Jani Tiainen]

9.1.2013 12:28, Ian kirjoitti:

On Wednesday, January 9, 2013 12:38:28 AM UTC-7, Jani Tiainen wrote:

Tested against latest master. Same behaviour.

In Oracle backend base.py is following piece of code:

# Check whether cx_Oracle was compiled with the WITH_UNICODE option.
This will
# also be True in Python 3.0.
if int(Database.version.split('.', 1)[0]) >= 5 and not
hasattr(Database,
'UNICODE'):
  convert_unicode = force_text
else:
  convert_unicode = force_bytes

Which was added in
>

Thing is that my cx_Oracle is version 5.1.2, it has cx_Oracle.UNICODE
definition.


That sounds correct.  The cx_Oracle.UNICODE type constant is present
when cx_Oracle is compiled *without* the WITH_UNICODE option (which no
longer exists in 5.1 anyway).

And Django uses smart_str / force_bytes.

If I remove that and use convert_unicode as force_text / force_unicode
everything works as expected.


Strange, in 5.1 it shouldn't make any difference which is used, as long
as your NLS_LANG is getting set properly in the backend.  What is your
server setup?  It seems that sometimes that can get interfered with if
you have other services using Oracle in the same process.  It shouldn't
hurt anything though for us to do an additional check for cx_Oracle 5.1+
and always use force_text in that case.



Server is running Oracle Database 10g Release 10.2.0.5.0 - 64bit 
Production. (EE edition)


and charset info:
NLS_CHARACTERSETWE8ISO8859P1
NLS_NCHAR_CHARACTERSET  AL16UTF16

When cx_Oracle (Version 5.1.2) is compiled against 10.2.0.3 client:

I can insert unicode characters directly using cx_Oracle.
I can't insert unicode characters using ORM
I can't insert unicode characters using Django connection.cursor()

When cx_Oracle is compiled against instantclient 11.2 (multinational 
version) I can do all of the above without the problems.



--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Ian]

On Wednesday, January 9, 2013 12:38:28 AM UTC-7, Jani Tiainen wrote:
>
> Tested against latest master. Same behaviour. 
>
> In Oracle backend base.py is following piece of code: 
>
> # Check whether cx_Oracle was compiled with the WITH_UNICODE option. 
> This will 
> # also be True in Python 3.0. 
> if int(Database.version.split('.', 1)[0]) >= 5 and not hasattr(Database, 
> 'UNICODE'): 
>  convert_unicode = force_text 
> else: 
>  convert_unicode = force_bytes 
>
> Which was added in  
>
> Thing is that my cx_Oracle is version 5.1.2, it has cx_Oracle.UNICODE 
> definition. 
>

That sounds correct.  The cx_Oracle.UNICODE type constant is present when 
cx_Oracle is compiled *without* the WITH_UNICODE option (which no longer 
exists in 5.1 anyway).

 

> And Django uses smart_str / force_bytes. 
>
> If I remove that and use convert_unicode as force_text / force_unicode 
> everything works as expected. 
>

Strange, in 5.1 it shouldn't make any difference which is used, as long as 
your NLS_LANG is getting set properly in the backend.  What is your server 
setup?  It seems that sometimes that can get interfered with if you have 
other services using Oracle in the same process.  It shouldn't hurt 
anything though for us to do an additional check for cx_Oracle 5.1+ and 
always use force_text in that case. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/CIo27txyz84J.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Jani Tiainen]

Ok, found source of the problem - but I don't know the solution.

I'm using Oracle client 10.2.0.3.0. It seems that unicode doesn't work 
there.


I compiled cx_Oracle against 11g instantclient 11.2 and it worked just fine.

So it must be something that Django assumes with Oracle and unicode 
capability.


I had cx_Oracle.UNICODE defined always which is checked in the code. I 
don't really know why.



9.1.2013 8:56, Jani Tiainen kirjoitti:

8.1.2013 21:00, akaariai kirjoitti:

I created the following test case into django's test suite modeltests/
basic/tests.py:
 def test_unicode(self):
 # Note: from __future__ import unicode_literals is in
effect...
 a = Article.objects.create(headline='0
\u0442\u0435\u0441\u0442 test', pub_date=datetime.n  ow())
 self.assertEqual(Article.objects.get(pk=a.pk).headline, '0
\u0442\u0435\u0441\u0442 test'   )

This does pass on Oracle when using Django's master branch, both with
Python 2.7 and 3.3.

Django's backend is doing all sorts of trickery behind the scenes to
get correct unicode handling. I am not sure where the problem is. What
Django version are you using?


Sorry about forgotting version info. I tested with 1.3.1 and 1.4.1 and
both gave same behaviour.

And I know that there is quite a lot of trickery going on. I'll try to
figure out what causes that problem.


On 8 tammi, 17:34, Jani Tiainen  wrote:

Hi,

I've been trying to save UTF-8 characters to oracle database without
success.

I've verified that database is indeed UTF-8 capable.

I can insert UTF-8 characters directly using cx_Oracle.

But when I use ORM it will trash characters.

Model I use:

class MyTest(models.Model):
  txt = CharField(max_length=128)

s = u'0 \u0442\u0435\u0441\u0442 test'

i = MyTest()
i.txt = s
i.save()

i2 = MyTest.objects.get(id=i.id)
print i2.txt

u'0 \xbf\xbf\xbf\xbf test'

So what happens here? It looks like Django trashes my unicode string at
some (unknown point).

Additional note:

If I use cursor() from Django connection object strings get broken also.
So it must be django Oracle backend doing something evil for me.

--
Jani Tiainen

- Well planned is half done and a half done has been sufficient
before...








--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Jani Tiainen]

Tested against latest master. Same behaviour.

In Oracle backend base.py is following piece of code:

# Check whether cx_Oracle was compiled with the WITH_UNICODE option. 
This will

# also be True in Python 3.0.
if int(Database.version.split('.', 1)[0]) >= 5 and not hasattr(Database, 
'UNICODE'):

convert_unicode = force_text
else:
convert_unicode = force_bytes

Which was added in 

Thing is that my cx_Oracle is version 5.1.2, it has cx_Oracle.UNICODE 
definition.


And Django uses smart_str / force_bytes.

If I remove that and use convert_unicode as force_text / force_unicode 
everything works as expected.


9.1.2013 8:56, Jani Tiainen kirjoitti:

8.1.2013 21:00, akaariai kirjoitti:

I created the following test case into django's test suite modeltests/
basic/tests.py:
 def test_unicode(self):
 # Note: from __future__ import unicode_literals is in
effect...
 a = Article.objects.create(headline='0
\u0442\u0435\u0441\u0442 test', pub_date=datetime.n  ow())
 self.assertEqual(Article.objects.get(pk=a.pk).headline, '0
\u0442\u0435\u0441\u0442 test'   )

This does pass on Oracle when using Django's master branch, both with
Python 2.7 and 3.3.

Django's backend is doing all sorts of trickery behind the scenes to
get correct unicode handling. I am not sure where the problem is. What
Django version are you using?


Sorry about forgotting version info. I tested with 1.3.1 and 1.4.1 and
both gave same behaviour.

And I know that there is quite a lot of trickery going on. I'll try to
figure out what causes that problem.


On 8 tammi, 17:34, Jani Tiainen  wrote:

Hi,

I've been trying to save UTF-8 characters to oracle database without
success.

I've verified that database is indeed UTF-8 capable.

I can insert UTF-8 characters directly using cx_Oracle.

But when I use ORM it will trash characters.

Model I use:

class MyTest(models.Model):
  txt = CharField(max_length=128)

s = u'0 \u0442\u0435\u0441\u0442 test'

i = MyTest()
i.txt = s
i.save()

i2 = MyTest.objects.get(id=i.id)
print i2.txt

u'0 \xbf\xbf\xbf\xbf test'

So what happens here? It looks like Django trashes my unicode string at
some (unknown point).

Additional note:

If I use cursor() from Django connection object strings get broken also.
So it must be django Oracle backend doing something evil for me.

--
Jani Tiainen

- Well planned is half done and a half done has been sufficient
before...








--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[Jani Tiainen]

8.1.2013 21:00, akaariai kirjoitti:

I created the following test case into django's test suite modeltests/
basic/tests.py:
 def test_unicode(self):
 # Note: from __future__ import unicode_literals is in
effect...
 a = Article.objects.create(headline='0
\u0442\u0435\u0441\u0442 test', pub_date=datetime.n  ow())
 self.assertEqual(Article.objects.get(pk=a.pk).headline, '0
\u0442\u0435\u0441\u0442 test'   )

This does pass on Oracle when using Django's master branch, both with
Python 2.7 and 3.3.

Django's backend is doing all sorts of trickery behind the scenes to
get correct unicode handling. I am not sure where the problem is. What
Django version are you using?


Sorry about forgotting version info. I tested with 1.3.1 and 1.4.1 and 
both gave same behaviour.


And I know that there is quite a lot of trickery going on. I'll try to 
figure out what causes that problem.



On 8 tammi, 17:34, Jani Tiainen  wrote:

Hi,

I've been trying to save UTF-8 characters to oracle database without
success.

I've verified that database is indeed UTF-8 capable.

I can insert UTF-8 characters directly using cx_Oracle.

But when I use ORM it will trash characters.

Model I use:

class MyTest(models.Model):
  txt = CharField(max_length=128)

s = u'0 \u0442\u0435\u0441\u0442 test'

i = MyTest()
i.txt = s
i.save()

i2 = MyTest.objects.get(id=i.id)
print i2.txt

u'0 \xbf\xbf\xbf\xbf test'

So what happens here? It looks like Django trashes my unicode string at
some (unknown point).

Additional note:

If I use cursor() from Django connection object strings get broken also.
So it must be django Oracle backend doing something evil for me.

--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...





--
Jani Tiainen

- Well planned is half done and a half done has been sufficient before...

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM, Oracle and UTF-8 encoding problem.

[akaariai]

I created the following test case into django's test suite modeltests/
basic/tests.py:
def test_unicode(self):
# Note: from __future__ import unicode_literals is in
effect...
a = Article.objects.create(headline='0
\u0442\u0435\u0441\u0442 test', pub_date=datetime.n  ow())
self.assertEqual(Article.objects.get(pk=a.pk).headline, '0
\u0442\u0435\u0441\u0442 test'   )

This does pass on Oracle when using Django's master branch, both with
Python 2.7 and 3.3.

Django's backend is doing all sorts of trickery behind the scenes to
get correct unicode handling. I am not sure where the problem is. What
Django version are you using?

 - Anssi

On 8 tammi, 17:34, Jani Tiainen  wrote:
> Hi,
>
> I've been trying to save UTF-8 characters to oracle database without
> success.
>
> I've verified that database is indeed UTF-8 capable.
>
> I can insert UTF-8 characters directly using cx_Oracle.
>
> But when I use ORM it will trash characters.
>
> Model I use:
>
> class MyTest(models.Model):
>      txt = CharField(max_length=128)
>
> s = u'0 \u0442\u0435\u0441\u0442 test'
>
> i = MyTest()
> i.txt = s
> i.save()
>
> i2 = MyTest.objects.get(id=i.id)
> print i2.txt
>
> u'0 \xbf\xbf\xbf\xbf test'
>
> So what happens here? It looks like Django trashes my unicode string at
> some (unknown point).
>
> Additional note:
>
> If I use cursor() from Django connection object strings get broken also.
> So it must be django Oracle backend doing something evil for me.
>
> --
> Jani Tiainen
>
> - Well planned is half done and a half done has been sufficient before...

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM only/defer calls do not work on cross-table relations

[John]

Works perfectly, thanks :)  Tested with a join 4 levels deep too and
it made just one query.

I use select_related elsewhere for more generic optimizations but it
hadn't occurred to me that django would need the hint once in
conjunction with an only() call.  Perhaps a patch to ensure that the
appropriate select_related call is made in conjunction with an only()
would make sense?

On Sep 27, 8:28 pm, Alasdair Nicol  wrote:
> Hi John,
>
> Use select_related [1] to tell Django to 'follow' the foreign key. Try
> the following:
>
> testmodels = 
> models.ATestModel.objects.all().select_related('other').only('other__name')
> print testmodels[0].other.name
>
> Regards,
> Alasdair
>
> [1]:https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-re...
>
> On 27/09/11 21:49, John wrote:
>
>
>
>
>
>
>
>
>
> > Hey,
>
> > I'm trying to improve the performance of a Django app, and noticed
> > that you can't seem to properly defer fields when making lookups with
> > joins.  To demonstrate, I set up a test project with the following
> > models:
>
> > class ATestModel(models.Model):
> >      other = models.ForeignKey('OtherModel')
>
> > class OtherModel(models.Model):
> >      name = models.CharField(max_length=32)
>
> > then, in the Django shell, I set up 10 'ATestModel's and two
> > 'OtherModel's properly linked up, and ran the following:
>
>  from testmodule import models
>  testmodels = models.ATestModel.objects.all().only('other__name')
>  print testmodels[0].other.name
> > Test One
>  from django.db import connection
>  print '\n\n'.join([x['sql'] for x in connection.queries])
> > SELECT "testmodule_atestmodel"."id",
> > "testmodule_atestmodel"."other_id" FROM "testmodule_atestmodel" LIMIT
> > 1
>
> > SELECT "testmodule_othermodel"."id", "testmodule_othermodel"."name"
> > FROM "testmodule_othermodel" WHERE "testmodule_othermodel"."id" = 1
>
> > I also re-ran that without the 'only' call:
>
>  testmodels = models.ATestModel.objects.all()
>  print testmodels[0].other.name
> > Test One
>  print '\n\n'.join([x['sql'] for x in connection.queries[2:]])
> > SELECT "testmodule_atestmodel"."id",
> > "testmodule_atestmodel"."other_id" FROM "testmodule_atestmodel" LIMIT
> > 1
>
> > SELECT "testmodule_othermodel"."id", "testmodule_othermodel"."name"
> > FROM "testmodule_othermodel" WHERE "testmodule_othermodel"."id" = 1
>
> > The 'only' call does nothing, and two queries are made when one would
> > have sufficed (SELECT testmodule_othermodel.name FROM
> > testmodule_atestmodel LEFT JOIN testmodule_othermodel ON
> > testmodule_atestmodel.id = testmodule_othermodule.id).  This raises a
> > huge performance penalty for highly normalized schemas and I would
> > even go so far as to say it in fact makes it impossible to optimize
> > queries with a decently normalized schema.
>
> > My apologies if someone has brought this up before - I was unable to
> > see anything about it in the django tickets or mailing lists.
>
> --
> Alasdair Nicol
> Developer, MEMSET
>
> mail: alasd...@memset.com
>   web:http://www.memset.com/
>
> Memset Ltd., registration number 4504980. 25 Frederick Sanger Road, 
> Guildford, Surrey, GU2 7YD, UK.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM only/defer calls do not work on cross-table relations

[Alasdair Nicol]

Hi John,

Use select_related [1] to tell Django to 'follow' the foreign key. Try 
the following:


testmodels = 
models.ATestModel.objects.all().select_related('other').only('other__name')
print testmodels[0].other.name


Regards,
Alasdair

[1]: 
https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related


On 27/09/11 21:49, John wrote:

Hey,

I'm trying to improve the performance of a Django app, and noticed
that you can't seem to properly defer fields when making lookups with
joins.  To demonstrate, I set up a test project with the following
models:

class ATestModel(models.Model):
 other = models.ForeignKey('OtherModel')

class OtherModel(models.Model):
 name = models.CharField(max_length=32)

then, in the Django shell, I set up 10 'ATestModel's and two
'OtherModel's properly linked up, and ran the following:


from testmodule import models
testmodels = models.ATestModel.objects.all().only('other__name')
print testmodels[0].other.name

Test One

from django.db import connection
print '\n\n'.join([x['sql'] for x in connection.queries])

SELECT "testmodule_atestmodel"."id",
"testmodule_atestmodel"."other_id" FROM "testmodule_atestmodel" LIMIT
1

SELECT "testmodule_othermodel"."id", "testmodule_othermodel"."name"
FROM "testmodule_othermodel" WHERE "testmodule_othermodel"."id" = 1

I also re-ran that without the 'only' call:


testmodels = models.ATestModel.objects.all()
print testmodels[0].other.name

Test One

print '\n\n'.join([x['sql'] for x in connection.queries[2:]])

SELECT "testmodule_atestmodel"."id",
"testmodule_atestmodel"."other_id" FROM "testmodule_atestmodel" LIMIT
1

SELECT "testmodule_othermodel"."id", "testmodule_othermodel"."name"
FROM "testmodule_othermodel" WHERE "testmodule_othermodel"."id" = 1

The 'only' call does nothing, and two queries are made when one would
have sufficed (SELECT testmodule_othermodel.name FROM
testmodule_atestmodel LEFT JOIN testmodule_othermodel ON
testmodule_atestmodel.id = testmodule_othermodule.id).  This raises a
huge performance penalty for highly normalized schemas and I would
even go so far as to say it in fact makes it impossible to optimize
queries with a decently normalized schema.

My apologies if someone has brought this up before - I was unable to
see anything about it in the django tickets or mailing lists.




--
Alasdair Nicol
Developer, MEMSET

mail: alasd...@memset.com
 web: http://www.memset.com/

Memset Ltd., registration number 4504980. 25 Frederick Sanger Road, Guildford, 
Surrey, GU2 7YD, UK.

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM help

[Carl Meyer]

Hi Daniel,

On Apr 22, 5:57 pm, Daniel Gerzo  wrote:
> I have a following models:
>
> class Movie(models.Model):
>      title = models.CharField(max_length=255)
>      year = models.IntegerField(max_length=4)
>      rating = models.DecimalField(max_digits=2, decimal_places=1, default=0)
>
> class Request(models.Model):
>      movie = models.ForeignKey(Movie)
>      user = models.ForeignKey(User)
>      language = models.CharField(max_length=7, db_index=True)
>      notes = models.CharField(max_length=255, blank=True, null=True)
>      added = models.DateTimeField(default=datetime.now)
>
> I want to get all Requests, with count per each pair of Movie and language.
>
> So far I got this:
> Request.objects.values('movie', 'language').annotate(Count('language'))
>
> That seems to give me almost what I want, but that way I get a
> ValueQuerySet. Is there a way to get or convert this to a QuerySet? I'd
> like to get an easy way to access Movie attributes from the above query
> as I will need to display movie title and rating in the template for
> each result from the above query.
>
> So, is there some better way of what am I trying to accomplish? If not,
> how should I proceed in retrieving the Movie attributes? Will I need to
> fetch those with something like Movie.objects.filter(pk__in=[list of
> ids]) and then manually map those?

That's one way, if you actually get full Movie objects out. However,
if you just need certain fields from the movie, you should be able to
do this:

Request.objects.values("movie__title",
"language").annotate(Count("language"))

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: orm: operations between models fields

[Tomasz Zieliński]

On 19 Mar, 22:49, maxi  wrote:
> Hi,
>
> Which is the better way to do this on orm language ?
>
> select filed1, sum(field2 * field3)
> from a_table
> group by field1
>

I believe this is not possible in the current Django ORM.
It would need something like this to work:

   MyModel.objects.annotate(mult_result=F('field2') *
F('field3')).values('field1').annotate(sum_result=Sum('mult_result'))

- which I find much more ugly than the raw SQL.

Also, you doesn't seem to need to get database rows (objects), just
some custom statistics, and ORM stands for Object-Relational Mapper,
which also makes the SQL way perfectly justified.

--
Tomasz Zielinski
pyconsultant.eu

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM Query question

[Tim Shaffer]

Or, using range:

MyModel.objects.filter( Q(a__range=(1,5)) | Q(b__range=(20,70)) )

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM Query question

[Tim Shaffer]

http://docs.djangoproject.com/en/dev/topics/db/queries/#complex-lookups-with-q-objects

MyModel.objects.filter( ( Q(a__gt=1) & Q(a__lt=5) ) | ( Q(b__gt=20) &
Q(b__lt=70) ) )

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM Query question

[rebus_]

On 13 April 2010 01:09, zweb  wrote:
> how do i do this kind of condition in django orm filter:
>
> ( 1 < a < 5)  or ( 20 < b < 70)
>
> select * from table where (a between 1 and 5) or (b between 20 and 70)
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
>
>

http://docs.djangoproject.com/en/dev/ref/models/querysets/#range

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM: get, filter and DoesNotExist

[Russell Keith-Magee]

On Mon, Apr 5, 2010 at 11:33 AM, zweb  wrote:
> after some debugging, just found out that
> filter method does not throw DoesNotExist while get does.

get() returns a single object. If a single object does not exist, you
get a DoesNotExist exception. This is documented:

http://docs.djangoproject.com/en/dev/ref/models/querysets/#id5

filter() applies a filtering condition to return a QuerySet of objects
that match the filter. If no objects match the filter, the queryset
can be empty. The result exists - it just doesn't contain anything
(e.g., the set of "all people that are 4m tall" exists, but has no
members). This is also documented:

http://docs.djangoproject.com/en/dev/ref/models/querysets/#filter-kwargs

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM using tons of memory and CPU

[Jirka Vejrazka]

Well, I was bound to get something wrong :)

> to_be_purged = 
> archivedEmail.objects.filter(received__lte=newest_to_delete).values('cacheID',
>  flat=True)

the end of the line should be  .values_list('cacheID', flat=True)   #
not .values(

  Cheers

Jirka

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM using tons of memory and CPU

[Jirka Vejrazka]

Hi,

 correct me if I got it wrong, but you essentially need these 4 things:
 1) obtain the date for the the newest messages to delete
 2) get cacheID of all objects to be deleted
 3) delete the files
 4) delete these objects from the database

So, you could use something like this:

# get the date
newest_to_delete = datetime.today() - timedelta(days=settings.DAYSTOKEEP)
# get cacheID's of emails to be deleted, requires Django 1.0+
to_be_purged = 
archivedEmail.objects.filter(received__lte=newest_to_delete).values('cacheID',
flat=True)
# to_be_purged is now a Python list of cacheID's
for item in to_be_purged:
delete_file(item)  # do the os.unlink() operations
# now delete the entries from the database
archivedEmail.objects.filter(received__lte=newest_to_delete).delete()

Since you keep all files in flat directories, I'm assuming that you
are not deleting millions of emails per day (otherwise the file lookup
operations would really be the bottleneck). Hence, this "list of
cacheID's" approach might be the fastest way. The iterator idea
suggested before is good too, you'd have to compare them yourself on
your data set to figure out what works better.

  Hope this helps

Jirka

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM using tons of memory and CPU

[Tracy Reed]

On Tue, Dec 15, 2009 at 03:35:29AM -0800, bruno desthuilliers spake thusly:
> looks like settings.DEBUG=True to me.

Nope. settings.py has DEBUG = False

> wrt/ the other mentioned problem - building whole model instances for
> each row - you can obviously save a lot of work here by using a
> value_list queryset - tuples are very cheap.

I don't understand which other problem you are referring to here... If
I mentioned something about building whole model instances for each
row I didn't realize it.

> Now for something different - here are a couple other python
> optimisation tricks:

Thanks!

> Oh and yes, one last point : how do you run this script exactly ?

I set the PYTHONPATH and DJANGO_SETTINGS_MODULE env vars and then do 
./purgemail.py

-- 
Tracy Reed
http://tracyreed.org


pgpSPr64Iuvkk.pgp
Description: PGP signature

Re: ORM using tons of memory and CPU

[Tracy Reed]

On Tue, Dec 15, 2009 at 09:43:02AM +0200, Jani Tiainen spake thusly:
> If you have DEBUG=True setting Django also records _every_ SQL query
> made to database and depending on a case, it might use quite lot of
> memory.

My settings.py contains:

DEBUG = False

-- 
Tracy Reed
http://tracyreed.org


pgp8LdeChJimo.pgp
Description: PGP signature

Re: ORM using tons of memory and CPU

[bruno desthuilliers]

On 15 déc, 02:44, Tracy Reed  wrote:
> I have code which looks basically like this:
>
> now        = datetime.today()
> beginning  = datetime.fromtimestamp(0)
> end        = now - timedelta(days=settings.DAYSTOKEEP)
>
> def purgedb():
>     """Delete archivedEmail objects from the beginning of time until
>     daystokeep days in the past."""
>     queryset   = archivedEmail.objects.all()
>     purgeset   = queryset.filter(received__range=(beginning, end))
>     for email in purgeset:
>         print email
>         try:
>             os.unlink(settings.REAVER_CACHE+"texts/%s"     % email.cacheID)
>             os.unlink(settings.REAVER_CACHE+"prob_good/%s" % email.cacheID)
>             os.unlink(settings.REAVER_CACHE+"prob_spam/%s" % email.cacheID)
>         except OSError:
>             pass
>     purgeset.delete()
>
> if __name__ == '__main__':
>     purgedb()
>
(snip)

> But when purgedb runs it deletes emails 100 at a time (which takes
> forever) and after running for a couple of hours uses a gig and a half
> of RAM. If I let it continue after a number of hours it runs the
> machine out of RAM/swap.

looks like settings.DEBUG=True to me.

> Am I doing something which is not idiomatic or misusing the ORM
> somehow? My understanding is that it should be lazy so using
> objects.all() on queryset and then narrowing it down with a
> queryset.filter() to make a purgeset should be ok, right?

No problem here as long as you don't do anything that forces
evaluation of the queryset. But this is still redundant - you can as
well build the appropriate queryset immediatly.

> What can I
> do to make this run in reasonable time/memory?

Others already commented on checking whether you have settings.DEBUG
set to True - the usual suspects when it comes to RAM issues with
django's ORM.

wrt/ the other mentioned problem - building whole model instances for
each row - you can obviously save a lot of work here by using a
value_list queryset - tuples are very cheap.

Oh, and yes: I/O and filesystem operations are not free neither. This
doesn't solve your pb with the script eating all the RAM, but surely
impacts the overall performances.


Now for something different - here are a couple other python
optimisation tricks:

> for email in purgeset:
> print email

Remove this. I/O are not for free. Really.

> try:
> os.unlink(settings.REAVER_CACHE+"texts/%s" % email.cacheID)
> os.unlink(settings.REAVER_CACHE+"prob_good/%s" % email.cacheID)
> os.unlink(settings.REAVER_CACHE+"prob_spam/%s" % email.cacheID)
> except OSError:
> pass

Move all redundant attribute lookup (os.unlink and
settings.REAVER_CACHE) and string concatenations out of this loop.


def purgedb():
  """Delete archivedEmail objects from the beginning of time until
daystokeep days in the past.
  """
  text_cache = settings.REAVER_CACHE + "texts/%s"
  prob_good_cache = settings.REAVER_CACHE+"prob_good/%s"
  prob_spam_cache = settings.REAVER_CACHE+"prob_spam/%s"
  unlink = os.unlink

  # no reason to put this outside the function.
  now = datetime.today()
  beginning = datetime.fromtimestamp(0)
  end = now - timedelta(days=settings.DAYSTOKEEP)
  qs = archivedEmail.objects.filter(received__range=(beginning, end))

  for row in qs.value_list(cacheID):
cacheID = row[0]
try:
  unlink(text_cache % cacheID)
  unlink(prob_good_cache % cacheID)
  unlink(prob_spam_cache % cacheID)
except OSError:
  pass

  qs.delete()

Oh and yes, one last point : how do you run this script exactly ?

HTH

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM using tons of memory and CPU

[rebus_]

2009/12/15 Tracy Reed :
>
> I have code which looks basically like this:
>
> now        = datetime.today()
> beginning  = datetime.fromtimestamp(0)
> end        = now - timedelta(days=settings.DAYSTOKEEP)
>
> def purgedb():
>    """Delete archivedEmail objects from the beginning of time until
>    daystokeep days in the past."""
>    queryset   = archivedEmail.objects.all()
>    purgeset   = queryset.filter(received__range=(beginning, end))

You don't need both queries (altghou they are lazy). You could just say:

purgeset = archivedEmail.filter(received__range=(beginning, end))


>    for email in purgeset:
>        print email
>        try:
>            os.unlink(settings.REAVER_CACHE+"texts/%s"     % email.cacheID)
>            os.unlink(settings.REAVER_CACHE+"prob_good/%s" % email.cacheID)
>            os.unlink(settings.REAVER_CACHE+"prob_spam/%s" % email.cacheID)
>        except OSError:
>            pass
>    purgeset.delete()
>
> if __name__ == '__main__':
>    purgedb()
>
> The idea is that we are stuffing a bunch of emails in a database for
> customer service purposes. I want to clear out anything older than
> DAYSTOKEEP. The model looks like this:
>
> class archivedEmail(models.Model):
>    subject     = models.CharField(blank=True, max_length=512, null=True)
>    toAddress   = models.CharField(blank=True, max_length=128, db_index=True)
>    fromAddress = models.CharField(blank=True, max_length=128, db_index=True)
>    date        = models.DateTimeField()
>    received    = models.DateTimeField(db_index=True)
>    crmScore    = models.FloatField()
>    spamStatus  = models.CharField(max_length=6, choices=spamStatusChoices, 
> db_index=True)
>    cacheHost   = models.CharField(max_length=24)
>    cacheID     = models.CharField(max_length=31, primary_key=True)
>
>    class Meta:
>        ordering = ('-received',)
>
> But when purgedb runs it deletes emails 100 at a time (which takes
> forever) and after running for a couple of hours uses a gig and a half
> of RAM. If I let it continue after a number of hours it runs the
> machine out of RAM/swap.
>
> Am I doing something which is not idiomatic or misusing the ORM
> somehow? My understanding is that it should be lazy so using
> objects.all() on queryset and then narrowing it down with a
> queryset.filter() to make a purgeset should be ok, right? What can I
> do to make this run in reasonable time/memory?
>
> PS: I used to have ordering set to -date in the class Meta but that
> caused the db to always put an ORDER BY date on the select query which
> was unnecessary in this case causing it to take ages sorting a couple
> million rows since there is no index on date (nor did there need to
> be, so I thought, since we never select on it). Changing it to
> received makes no difference to my app but avoids creating another
> index. Django's is the first ORM I have ever used and these sneaky
> performance issues are making me wonder...
>
> --
> Tracy Reed
> http://tracyreed.org
>


When you execute a query set that operates on millions of rows you
should remember that ORM will create a python object for each row.
This could take a while. Though i am not sure how it handles memory
issues.

You could issue plain SQL query which returns result as tuple (only
returning cache ID for example) and might be faster if you don't
really need ORM, then loop through the tuple and delete files on disk.
I think this should use less memory and processing time.

Also, I am not SQL guru but i guess BETWEEN should work on most DB
servers so you should not have portability issues (but i cant
guarantee this).

Don't use try ... except for checking if the file exists. Rather use
"if" statement with os.path.exists or os.path.isfile and if it does
exist delete the file. Exceptions are expensive in CPython.

Read this topic about deleting objects in bulk (though i think you
already have it never hurts to refresh your memory).

http://docs.djangoproject.com/en/dev/topics/db/queries/#deleting-objects

Just my 2 cents

Davor

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM using tons of memory and CPU

[Jani Tiainen]

On Mon, 2009-12-14 at 17:44 -0800, Tracy Reed wrote:
> I have code which looks basically like this:
> 
> now= datetime.today()
> beginning  = datetime.fromtimestamp(0)
> end= now - timedelta(days=settings.DAYSTOKEEP)
> 
> def purgedb():
> """Delete archivedEmail objects from the beginning of time until
> daystokeep days in the past."""
> queryset   = archivedEmail.objects.all()
> purgeset   = queryset.filter(received__range=(beginning, end))
> for email in purgeset:
> print email
> try:
> os.unlink(settings.REAVER_CACHE+"texts/%s" % email.cacheID)
> os.unlink(settings.REAVER_CACHE+"prob_good/%s" % email.cacheID)
> os.unlink(settings.REAVER_CACHE+"prob_spam/%s" % email.cacheID)
> except OSError:
> pass
> purgeset.delete()
> 
> if __name__ == '__main__':
> purgedb()
> 
> The idea is that we are stuffing a bunch of emails in a database for
> customer service purposes. I want to clear out anything older than
> DAYSTOKEEP. The model looks like this:
> 
> class archivedEmail(models.Model):
> subject = models.CharField(blank=True, max_length=512, null=True)
> toAddress   = models.CharField(blank=True, max_length=128, db_index=True)
> fromAddress = models.CharField(blank=True, max_length=128, db_index=True)
> date= models.DateTimeField()
> received= models.DateTimeField(db_index=True)
> crmScore= models.FloatField()
> spamStatus  = models.CharField(max_length=6, choices=spamStatusChoices, 
> db_index=True)
> cacheHost   = models.CharField(max_length=24)
> cacheID = models.CharField(max_length=31, primary_key=True)
> 
> class Meta:
> ordering = ('-received',)
> 
> But when purgedb runs it deletes emails 100 at a time (which takes
> forever) and after running for a couple of hours uses a gig and a half
> of RAM. If I let it continue after a number of hours it runs the
> machine out of RAM/swap.
> 
> Am I doing something which is not idiomatic or misusing the ORM
> somehow? My understanding is that it should be lazy so using
> objects.all() on queryset and then narrowing it down with a
> queryset.filter() to make a purgeset should be ok, right? What can I
> do to make this run in reasonable time/memory?
> 
> PS: I used to have ordering set to -date in the class Meta but that
> caused the db to always put an ORDER BY date on the select query which
> was unnecessary in this case causing it to take ages sorting a couple
> million rows since there is no index on date (nor did there need to
> be, so I thought, since we never select on it). Changing it to
> received makes no difference to my app but avoids creating another
> index. Django's is the first ORM I have ever used and these sneaky
> performance issues are making me wonder...
> 

If you have DEBUG=True setting Django also records _every_ SQL query
made to database and depending on a case, it might use quite lot of
memory.

-- 

Jani Tiainen


--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM using tons of memory and CPU

[fordprefect]

Try using a) queryset.iterator() to iterate through the results and b)
paginating the results. Otherwise you run the risk of loading all the
results into memory at once,

On Dec 15, 1:44 am, Tracy Reed  wrote:
> I have code which looks basically like this:
>
> now        = datetime.today()
> beginning  = datetime.fromtimestamp(0)
> end        = now - timedelta(days=settings.DAYSTOKEEP)
>
> def purgedb():
>     """Delete archivedEmail objects from the beginning of time until
>     daystokeep days in the past."""
>     queryset   = archivedEmail.objects.all()
>     purgeset   = queryset.filter(received__range=(beginning, end))
>     for email in purgeset:
>         print email
>         try:
>             os.unlink(settings.REAVER_CACHE+"texts/%s"     % email.cacheID)
>             os.unlink(settings.REAVER_CACHE+"prob_good/%s" % email.cacheID)
>             os.unlink(settings.REAVER_CACHE+"prob_spam/%s" % email.cacheID)
>         except OSError:
>             pass
>     purgeset.delete()
>
> if __name__ == '__main__':
>     purgedb()
>
> The idea is that we are stuffing a bunch of emails in a database for
> customer service purposes. I want to clear out anything older than
> DAYSTOKEEP. The model looks like this:
>
> class archivedEmail(models.Model):
>     subject     = models.CharField(blank=True, max_length=512, null=True)
>     toAddress   = models.CharField(blank=True, max_length=128, db_index=True)
>     fromAddress = models.CharField(blank=True, max_length=128, db_index=True)
>     date        = models.DateTimeField()
>     received    = models.DateTimeField(db_index=True)
>     crmScore    = models.FloatField()
>     spamStatus  = models.CharField(max_length=6, choices=spamStatusChoices, 
> db_index=True)
>     cacheHost   = models.CharField(max_length=24)
>     cacheID     = models.CharField(max_length=31, primary_key=True)
>
>     class Meta:
>         ordering = ('-received',)
>
> But when purgedb runs it deletes emails 100 at a time (which takes
> forever) and after running for a couple of hours uses a gig and a half
> of RAM. If I let it continue after a number of hours it runs the
> machine out of RAM/swap.
>
> Am I doing something which is not idiomatic or misusing the ORM
> somehow? My understanding is that it should be lazy so using
> objects.all() on queryset and then narrowing it down with a
> queryset.filter() to make a purgeset should be ok, right? What can I
> do to make this run in reasonable time/memory?
>
> PS: I used to have ordering set to -date in the class Meta but that
> caused the db to always put an ORDER BY date on the select query which
> was unnecessary in this case causing it to take ages sorting a couple
> million rows since there is no index on date (nor did there need to
> be, so I thought, since we never select on it). Changing it to
> received makes no difference to my app but avoids creating another
> index. Django's is the first ORM I have ever used and these sneaky
> performance issues are making me wonder...
>
> --
> Tracy Reedhttp://tracyreed.org
>
>  application_pgp-signature_part
> < 1KViewDownload

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM

[Geobase Isoscale]

Hi all,
Thanks a lot, for the comments. I'm working with PostgreSQL database
together with with Django and GeoDjango. I intended to use views to query
externally joined tables that keep changing based on the updates. I was also
looking at database consistency by taking into account constraints with
triggers being part them. Thats how the discussion on triggers and views
came about.

The other issue is if someone is aware of modelling tools like UML does,
 that converts conceptual design (graphical classes) into django model
classes can forward me their names and user guides.


Many Thanks

Isoscale

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Mark (Nosrednakram)]

Hello Isoscale,

>  I intend to write ORM code that will create views and triggers in the
> database? Which parts of the source code should I alter?

I use django to access views commonly, primarily for reporting I'll
create views taking care of the complex joins from our ERP so
developers can easily use the ORM.  There are a few things to keep in
mind if you want to use the ORM to access views especially if they are
based on external tables.

  1. ORM expects tables with a single PK
  2. Don't add views into your models.py use another file to define
them

The first one is very important you'll need to come up with a single
unique identifier for each row which can be a concatenated string etc.
If you are using oracle you can use rownum see notes at . Can you explain the your purpose
that requires you to create views and triggers.

Hope this helps,
Mark


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM Cache in a standalone script

[Alexis MINEAUD]

Thanks a lot Karen, transaction was the problem. Resolved by :
connection.cursor().execute('set transaction isolation level read
committed')

Have a nice day :)


On Thu, Oct 8, 2009 at 1:52 PM, Karen Tracey  wrote:

> On Thu, Oct 8, 2009 at 5:07 AM, Alexis MINEAUD  wrote:
>
>> Ok, my bad, the  sequence works well, i just confused the field name...
>> But the problem is still there if the update is done by another actor than
>> Django itself.
>>
>> My standalone script is daemon which poll my DB with a XX.objects.all().
>> If i updated myself a row from XX, the daemon doesn't see the modification
>> until i relaunch it.
>>
>> Even a Tag.objects.get(id = 1) for example ignore the update.
>>
>> Is there a way to force the query again ?
>>
>> Django does issue the query again, if you are getting the same result it
> is because that is what the database is returning. You don't mention what
> database you are using. When I have seen this before it has always been due
> to MySQL/InnoDB's default transaction isolation level of repeatable read. If
> that is the DB you are using you might want to read this thread:
>
>
> http://groups.google.com/group/django-users/browse_thread/thread/e25cec400598c06d/
>
> If you are using a different DB you might want to investigate its
> trasnaction isolation level handling.
>
> Karen
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM Cache in a standalone script

[Karen Tracey]

On Thu, Oct 8, 2009 at 5:07 AM, Alexis MINEAUD  wrote:

> Ok, my bad, the  sequence works well, i just confused the field name...
> But the problem is still there if the update is done by another actor than
> Django itself.
>
> My standalone script is daemon which poll my DB with a XX.objects.all(). If
> i updated myself a row from XX, the daemon doesn't see the modification
> until i relaunch it.
>
> Even a Tag.objects.get(id = 1) for example ignore the update.
>
> Is there a way to force the query again ?
>
> Django does issue the query again, if you are getting the same result it is
because that is what the database is returning. You don't mention what
database you are using. When I have seen this before it has always been due
to MySQL/InnoDB's default transaction isolation level of repeatable read. If
that is the DB you are using you might want to read this thread:

http://groups.google.com/group/django-users/browse_thread/thread/e25cec400598c06d/

If you are using a different DB you might want to investigate its
trasnaction isolation level handling.

Karen

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Matt Schinckel]

On Oct 8, 6:18 pm, Marek Pietrucha  wrote:
> I see that all of you guy's know what your talking about. I was
> thinking way won't you share some knowledge to this 
> topic:http://groups.google.com/group/django-users/browse_thread/thread/1517...
>

What does this have to do with this thread?  You have asked the
question, asking it again isn't likely to (at least this soon) result
in more response.

Matt.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM Cache in a standalone script

[Alexis MINEAUD]

Ok, my bad, the  sequence works well, i just confused the field name...
But the problem is still there if the update is done by another actor than
Django itself.

My standalone script is daemon which poll my DB with a XX.objects.all(). If
i updated myself a row from XX, the daemon doesn't see the modification
until i relaunch it.

Even a Tag.objects.get(id = 1) for example ignore the update.

Is there a way to force the query again ?





On Thu, Oct 8, 2009 at 2:05 AM, Karen Tracey  wrote:

> On Wed, Oct 7, 2009 at 7:39 PM, laligatz  wrote:
>
>>
>> Hi everybody.
>>
>> I'm stuck on a problem with the Django ORM.
>> After a basic query like select * from table where id = 1, the result
>> is still the same although i've update the row in the DB.
>>
>> Example:
>>
>> >>> Tag.objects.all()
>> []
>>
>> >>> tag = Tag.objects.all().get()
>> >>> tag
>> 
>>
>> >>>  tag.name = 'tag11'
>> >>>  tag.save()
>>
>> >>>  Tag.objects.all()
>> []
>>
>> How to say to Django to make the query again ?
>>
>>
> I cannot recreate this.  With this model:
>
> class Tag(models.Model):
>name = models.CharField(max_length=22)
>def __unicode__(self):
>   return self.name
>
> your sequence of commands produces:
>
> >>> from ttt.models import Tag
> >>> Tag.objects.all()
> []
> >>> Tag.objects.create(name='tag1')
> 
> >>> Tag.objects.all()
> []
> >>> tag = Tag.objects.get()
> >>> tag
> 
> >>> tag.name = 'tag11'
> >>> tag.save()
> >>> Tag.objects.all()
> []
>
> Is there something you've left out of your account that might be a clue?
> What's this standalone script mentioned in the subject?  Despite looking
> like your sequence is from a singe python shell session, is the update being
> done by a different shell/script than the query?  If so, and if you are
> using MySQL/InnoDB I suspect the problem is to do its default transaction
> isolation level.  But at this point point I'm leaping pretty high to a
> conclusion based on guesses about what you might be doing, not what you've
> actually said you are doing.
>
> Karen
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Marek Pietrucha]

I see that all of you guy's know what your talking about. I was
thinking way won't you share some knowledge to this topic:
http://groups.google.com/group/django-users/browse_thread/thread/1517053b51a1d7c8/

Please read and give some response.

best regards.

On Oct 7, 7:05 pm, Christophe Pettus  wrote:
> On Oct 7, 2009, at 2:30 AM, Geobase Isoscale wrote:
>
> >  I intend to write ORM code that will create views and triggers in  
> > the database? Which parts of the source code should I alter?
>
> Django can work just fine with views and triggers, but the Django ORM  
> layer will not create them for you.  Views tend to be extremely  
> database-specific, and triggers even more so.
>
> In the case of views, you can create them using the database's native  
> view creation mechanism, and then set up a Django model to access  
> them, just as if they were a table (in this case, you would not use  
> syncdb to create the tables).  If you are using PostgreSQL, you can  
> even set them up to be updatable using the rules system.  As long as  
> they can be used just like a table with standard SQL SELECT / INSERT /  
> DELETE, Django should be perfectly happy to access them.
>
> In the case of triggers, there's no reason that a database that Django  
> is accessing cannot use triggers.  The only thing to keep in mind is  
> that if a trigger modifies a database row that also exists in memory  
> as a model object instance, nothing will cause the model object  
> instance to automatically reflect the changes in the database row, so  
> you might have "cache" consistency problems.
>
> So, if what you are asking is, "Can I use the Django ORM to access a  
> database in which I have created views and triggers," the answer is  
> "yes, provided you understand potential interactions."  If the  
> question is, "Can the Django ORM create views and triggers in the  
> database through its API," the answer is no.
> --
> -- Christophe Pettus
>     x...@thebuild.com
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM Cache in a standalone script

[Karen Tracey]

On Wed, Oct 7, 2009 at 7:39 PM, laligatz  wrote:

>
> Hi everybody.
>
> I'm stuck on a problem with the Django ORM.
> After a basic query like select * from table where id = 1, the result
> is still the same although i've update the row in the DB.
>
> Example:
>
> >>> Tag.objects.all()
> []
>
> >>> tag = Tag.objects.all().get()
> >>> tag
> 
>
> >>>  tag.name = 'tag11'
> >>>  tag.save()
>
> >>>  Tag.objects.all()
> []
>
> How to say to Django to make the query again ?
>
>
I cannot recreate this.  With this model:

class Tag(models.Model):
   name = models.CharField(max_length=22)
   def __unicode__(self):
  return self.name

your sequence of commands produces:

>>> from ttt.models import Tag
>>> Tag.objects.all()
[]
>>> Tag.objects.create(name='tag1')

>>> Tag.objects.all()
[]
>>> tag = Tag.objects.get()
>>> tag

>>> tag.name = 'tag11'
>>> tag.save()
>>> Tag.objects.all()
[]

Is there something you've left out of your account that might be a clue?
What's this standalone script mentioned in the subject?  Despite looking
like your sequence is from a singe python shell session, is the update being
done by a different shell/script than the query?  If so, and if you are
using MySQL/InnoDB I suspect the problem is to do its default transaction
isolation level.  But at this point point I'm leaping pretty high to a
conclusion based on guesses about what you might be doing, not what you've
actually said you are doing.

Karen

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Christophe Pettus]

On Oct 7, 2009, at 2:30 AM, Geobase Isoscale wrote:
>  I intend to write ORM code that will create views and triggers in  
> the database? Which parts of the source code should I alter?

Django can work just fine with views and triggers, but the Django ORM  
layer will not create them for you.  Views tend to be extremely  
database-specific, and triggers even more so.

In the case of views, you can create them using the database's native  
view creation mechanism, and then set up a Django model to access  
them, just as if they were a table (in this case, you would not use  
syncdb to create the tables).  If you are using PostgreSQL, you can  
even set them up to be updatable using the rules system.  As long as  
they can be used just like a table with standard SQL SELECT / INSERT /  
DELETE, Django should be perfectly happy to access them.

In the case of triggers, there's no reason that a database that Django  
is accessing cannot use triggers.  The only thing to keep in mind is  
that if a trigger modifies a database row that also exists in memory  
as a model object instance, nothing will cause the model object  
instance to automatically reflect the changes in the database row, so  
you might have "cache" consistency problems.

So, if what you are asking is, "Can I use the Django ORM to access a  
database in which I have created views and triggers," the answer is  
"yes, provided you understand potential interactions."  If the  
question is, "Can the Django ORM create views and triggers in the  
database through its API," the answer is no.
--
-- Christophe Pettus
x...@thebuild.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Jani Tiainen]

Geobase Isoscale kirjoitti:
> I have to live with this reality, maybe in future we might  develop a 
> code to extend Django to support views and triiggers. 

I don't get the use case here. What is this need for adding views and 
triggers? What purpose they serve for ORM?

For database they're pretty handy to do business logic if there is data 
incoming form external resources that are not passed through ORM that 
can take care of all that mess but this can already be done with Django.

There is (standard) way to add arbitary SQL scripts that are ran after 
django tables and such are generated or it can be done quite easily 
manually.

Could you provide some examples what you're trying to achieve to shed 
some light here..?

-- 
Jani Tiainen

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Geobase Isoscale]

I have to live with this reality, maybe in future we might  develop a code
to extend Django to support views and triiggers.

Many Thanks

Isoscale

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Russell Keith-Magee]

On Wed, Oct 7, 2009 at 5:30 PM, Geobase Isoscale  wrote:
> Hi Everyone,
>  I intend to write ORM code that will create views and triggers in the
> database? Which parts of the source code should I alter?

This is the third time you've asked the same question in less than a
week. I answered the first time [1]; Daniel Roseman answered the
second time [2].

[1] 
http://groups.google.com/group/django-users/browse_thread/thread/88d08478478199b6
[2] 
http://groups.google.com/group/django-users/browse_thread/thread/dd1c601dd7f5e1d1

The answer hasn't changed. Django's ORM doesn't provide support for
views and triggers.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: ORM

[Jani Tiainen]

Geobase Isoscale kirjoitti:
> Hi Everyone, 
> 
>  I intend to write ORM code that will create views and triggers in the 
> database? Which parts of the source code should I alter?

None...

There already exists mechanism to run arbitary SQL after syncdb...

Or did you meant something else since I'm not sure what you mean by 
"views" and "triggers" and how they are related to ORM...

-- 
Jani Tiainen

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---