Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 7:49 PM, Malcolm Tredinnick wrote: > I'd be somewhat against this, I think. It's *very* easy to reuse > querysets and inadvertently cause extra database queries. Unless you're > using really huge querysets, the memory usage is not going to kill you. > Pulling back the huge number of results already uses a bunch of memory > and that's a property of the db wrapper. There's a multiplier involved > for creating Python objects. Speaking as someone who has (accidentally) brought down a beefy server by accidentally evaluating a reasonably large QuerySet, I'd say there's not a whole lot we can do without impacting usability in other, more vital-to-support scenarios. When we had our nasty server-crashing query (which thankfully never made it out of staging; that's what staging servers are for and why you should have one to test things before you ever think about deploying), just fetching the data from the DB -- no object instantiation at all -- was a significant drain. Actually trying to instantiate the model objects kicked the usage up even higher, of course, but it was mostly an interesting exercise in watching the memory spike move from place to place as the data worked its way from the DB to the Python process in which Django was running. (incidentally, the above sort of situation is one reason why a QuerySet limits itself to a certain number of objects displayed in __repr__; the real killer was that an error was being thrown, and as part of the Django debug page it was trying to print the __repr__ of a QuerySet of, IIRC, about half a million objects. A QuerySet doesn't try to do that anymore) -- "Bureaucrat Conrad, you are technically correct -- the best kind of correct." --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Warning on multiple calls to QuerySet.iterator() (was Re: Is this true. that django really takes a lot of memory?)
On Tue, 2009-02-17 at 19:25 -0600, Jeremy Dunck wrote:
> On Tue, Feb 17, 2009 at 7:13 PM, Malcolm Tredinnick
> wrote:
> ...
> >> if settings.DEBUG and self.prior_iteration:
> >>warnings.warn("dope!")
> >
> > This certainly sounds reasonable and doable today without any real
> > overhead. Go ahead and make a patch/ticket.
>
> OK.
>
> Do you think there should be a PerformanceWarning class, or just use
> the default UserWarning?
It should be blue! :-)
Slight preference for using a standard warning type at the moment.
Either UserWarning or RuntimeWarning. Only a slight preference, though.
Please yourself here.
Please don't ask me what to do about issuing multiple times, because I
was thinking about that over lunch just now and it may be fiddly.
Issuing a warning always sounds right, since it's commonly going to be a
property of a template that will cause this to happen. But if you use
that template and hit a problem, you're going to be swamped with
warnings. We really need a "once per template" option that obviously
doesn't exist in the warnings module. I might be over-thinking it,
though. "Always" is probably the right answer.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: views.py getting big
Sorry... will do next time On Feb 17, 8:09 pm, Malcolm Tredinnick wrote: > On Tue, 2009-02-17 at 16:44 -0800, Tasmania wrote: > > Hi there, > > I have one single application that has building up functionalities so > > views.py has grown and it's getting big. The application is running > > fast, but I'd like to partitionate views.py (i.e. putting some > > functions in a different file, etc) can I do that in Django? > > Please ask this question on the django-users mailing list. This list > (django-dev) is for the internal development of Django itself, not "how > do I use Django" questions. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: views.py getting big
Thanks Alex! On Feb 17, 8:08 pm, Alex Gaynor wrote: > On Tue, Feb 17, 2009 at 7:44 PM, Tasmania wrote: > > > Hi there, > > I have one single application that has building up functionalities so > > views.py has grown and it's getting big. The application is running > > fast, but I'd like to partitionate views.py (i.e. putting some > > functions in a different file, etc) can I do that in Django? > > > Thanks! > > > T > > a) Yes you can keep views wherever you want, but in keeping with the views > convention you could make views a directory and make multiple files there > (plus an __init__.py file) and map the urls to the new address correctly. > b) by the time you get to this point it usually indicates that you're doing > too much in one application, checkout James' video on making your apps > reusable:http://www.youtube.com/watch?v=A-S0tqpPga4 > > Alex > > -- > "I disapprove of what you say, but I will defend to the death your right to > say it." --Voltaire > "The people's good is the highest law."--Cicero --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 7:13 PM, Malcolm Tredinnick
wrote:
...
>> if settings.DEBUG and self.prior_iteration:
>>warnings.warn("dope!")
>
> This certainly sounds reasonable and doable today without any real
> overhead. Go ahead and make a patch/ticket.
OK.
Do you think there should be a PerformanceWarning class, or just use
the default UserWarning?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: #10290: grouping with extra selects produces invalid SQL
On Tue, Feb 17, 2009 at 5:54 PM, Malcolm Tredinnick
wrote:
>
> Hi Ian,
>
> On Tue, 2009-02-17 at 16:05 -0700, Ian Kelly wrote:
> [...]
>> The solution that I'm proposing is to use the extra select expression
>> itself in the group by, rather than the alias. This passes the tests
>> across all four included backends, and seems to work in general as
>> long as the expression is not a scalar subquery (in which case I think
>> a general solution would require advanced inspection of the
>> expression). But it bothers me a bit because it just feels fragile.
>
> I'm pretty tired today and under a pile of past-urgent things to do, so
> excuse the request for training wheels on this one... can you give a
> short SQL example of what you're talking about here (before vs. after).
No problem. Take the queryset
`Author.objects.extra(select={'sqrt_age':
'sqrt(age)'}).values('sqrt_age').annotate(Count('book'))`
Current SQL: SELECT (sqrt(age)) AS "SQRT_AGE",
COUNT("AGGREGATION_REGRESS_BOOK_AC81D"."BOOK_ID") AS "BOOK__COUNT"
FROM "AGGREGATION_REGRESS_AUTHOR" LEFT OUTER JOIN
"AGGREGATION_REGRESS_BOOK_AC81D" ON ("AGGREGATION_REGRESS_AUTHOR"."ID"
= "AGGREGATION_REGRESS_BOOK_AC81D"."AUTHOR_ID") GROUP BY "SQRT_AGE"
Proposed SQL: SELECT (sqrt(age)) AS "SQRT_AGE",
COUNT("AGGREGATION_REGRESS_BOOK_AC81D"."BOOK_ID") AS "BOOK__COUNT"
FROM "AGGREGATION_REGRESS_AUTHOR" LEFT OUTER JOIN
"AGGREGATION_REGRESS_BOOK_AC81D" ON ("AGGREGATION_REGRESS_AUTHOR"."ID"
= "AGGREGATION_REGRESS_BOOK_AC81D"."AUTHOR_ID") GROUP BY (sqrt(age))
Regards,
Ian
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, 2009-02-17 at 18:57 -0600, Jeremy Dunck wrote:
> On Tue, Feb 17, 2009 at 6:49 PM, Malcolm Tredinnick
> wrote:
> ...
> > I'd be somewhat against this, I think. It's *very* easy to reuse
> > querysets and inadvertently cause extra database queries.
> ...
> > we're trading memory
> > usage for speed and ease of use (and providing a way to improve the
> > former in "expert mode").
>
> Point taken.
>
> I wish there were some way to issue a warning if _result_cache is
> filled but __iter__ isn't used more than once. :-/
Possible. Requires relying on __del__ being called so that we know when
it's not being used any longer. I prefer your other option, however.
> I could imagine a warning being issued if the functionality offered by
> .iterator is used more than once. That might be a happy medium-- then
> I could use .iterator as my default coding practice, and be slapped
> when I iterate more than once after all.
>
> if settings.DEBUG and self.prior_iteration:
>warnings.warn("dope!")
This certainly sounds reasonable and doable today without any real
overhead. Go ahead and make a patch/ticket.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: views.py getting big
On Tue, 2009-02-17 at 16:44 -0800, Tasmania wrote: > Hi there, > I have one single application that has building up functionalities so > views.py has grown and it's getting big. The application is running > fast, but I'd like to partitionate views.py (i.e. putting some > functions in a different file, etc) can I do that in Django? Please ask this question on the django-users mailing list. This list (django-dev) is for the internal development of Django itself, not "how do I use Django" questions. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: views.py getting big
On Tue, Feb 17, 2009 at 7:44 PM, Tasmania wrote: > > Hi there, > I have one single application that has building up functionalities so > views.py has grown and it's getting big. The application is running > fast, but I'd like to partitionate views.py (i.e. putting some > functions in a different file, etc) can I do that in Django? > > Thanks! > > T > > > > a) Yes you can keep views wherever you want, but in keeping with the views convention you could make views a directory and make multiple files there (plus an __init__.py file) and map the urls to the new address correctly. b) by the time you get to this point it usually indicates that you're doing too much in one application, checkout James' video on making your apps reusable: http://www.youtube.com/watch?v=A-S0tqpPga4 Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." --Voltaire "The people's good is the highest law."--Cicero --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
views.py getting big
Hi there, I have one single application that has building up functionalities so views.py has grown and it's getting big. The application is running fast, but I'd like to partitionate views.py (i.e. putting some functions in a different file, etc) can I do that in Django? Thanks! T --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 6:49 PM, Malcolm Tredinnick
wrote:
...
> I'd be somewhat against this, I think. It's *very* easy to reuse
> querysets and inadvertently cause extra database queries.
...
> we're trading memory
> usage for speed and ease of use (and providing a way to improve the
> former in "expert mode").
Point taken.
I wish there were some way to issue a warning if _result_cache is
filled but __iter__ isn't used more than once. :-/
I could imagine a warning being issued if the functionality offered by
.iterator is used more than once. That might be a happy medium-- then
I could use .iterator as my default coding practice, and be slapped
when I iterate more than once after all.
if settings.DEBUG and self.prior_iteration:
warnings.warn("dope!")
?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Ready for checkin / review (for 1.1): [ORM-09] Manager.update_or_create() and Model.update().
One of the desired but lacking a champion features for 1.1 ([ORM-09], #3182) has been updated to work with r9844 with tests and docs, and improved as per the last comments on the ticket. I think this one is ready to go in, but I didn't want to set the ticket "ready for checkin" until someone else can take a look. http://code.djangoproject.com/ticket/3182 http://code.djangoproject.com/wiki/Version1.1Roadmap#maybe-features Cheers. Tai. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #10290: grouping with extra selects produces invalid SQL
Hi Ian, On Tue, 2009-02-17 at 16:05 -0700, Ian Kelly wrote: [...] > The solution that I'm proposing is to use the extra select expression > itself in the group by, rather than the alias. This passes the tests > across all four included backends, and seems to work in general as > long as the expression is not a scalar subquery (in which case I think > a general solution would require advanced inspection of the > expression). But it bothers me a bit because it just feels fragile. I'm pretty tired today and under a pile of past-urgent things to do, so excuse the request for training wheels on this one... can you give a short SQL example of what you're talking about here (before vs. after). Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, 2009-02-17 at 15:20 -0600, Jeremy Dunck wrote:
> On Tue, Feb 17, 2009 at 3:15 PM, Jeremy Dunck wrote:
> ...
> > {% for question in poll.questions.all.iterator %}
> > works just fine.
> >
>
> OK, last one from me.
>
> As a 2.0 wish, I'd like to make .iterator the default behavior, and
> the cached-version a special case. I realize this point is quite
> debatable.
I'd be somewhat against this, I think. It's *very* easy to reuse
querysets and inadvertently cause extra database queries. Unless you're
using really huge querysets, the memory usage is not going to kill you.
Pulling back the huge number of results already uses a bunch of memory
and that's a property of the db wrapper. There's a multiplier involved
for creating Python objects. Since we have a way to not use the caching
if somebody wants to optimise on that level and since doing that and
then doing a second database access is quite slow, we're trading memory
usage for speed and ease of use (and providing a way to improve the
former in "expert mode").
I really don't look forward to the five questions a day on django-users
about all the databae queries that are happening. I know you're only
talking about the mythical 2.0, but that doesn't change how people will
behave. I'm strongly in favour of keeping Django's primary audience as
experienced developers wanting to work faster, but we do have a large
non-experienced and even absolute beginner userbase, so simple things
that can save them a lot of time aren't to be dismissed out of hand.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: App Engine port
On Tue, 2009-02-17 at 02:58 -0800, Waldemar Kornewald wrote: > No comments on the more efficient perms/contenttypes implementation? If you were hoping for something from me, I don't have any time to think about it. I'm not in the business of predicting the future in a general sense, beyond things I've thought hard about and are working on directly. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #10290: grouping with extra selects produces invalid SQL
On Tue, Feb 17, 2009 at 5:01 PM, Russell Keith-Magee wrote: > Out of interest - where does Oracle fall on this one? Can you use > aliases in a HAVING or ORDER BY? Oracle only allows aliases in the ORDER BY clause. > >> If there are no concerns or potential gotchas with this approach, then >> I'll go ahead and commit this in a few days. > > My only major concern would be any potential performance hit from > double-computing the extra clause. Alex's suggestion to parameterize > this in the connection ops backend sounds like a good idea to me. > However, rather than just defining a boolean feature flag, my > preference would be to define a 'expand_extra()' function (or similar > - bikeshed builder gets to choose the colour) on the ops backend which > just returns the alias in the default case, but returns the full > expression under Oracle. I should probably do the same for alias usage > in HAVING/ORDER_BY clauses. That sounds like a good idea to me. I've already found one bug in my patch, which is that I didn't account for the duplication of extra select_params into the group by clause. Since we currently assume the group by clause has no params, fixing that will already be non-trivial. Regards, Ian --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #10290: grouping with extra selects produces invalid SQL
On Wed, Feb 18, 2009 at 8:05 AM, Ian Kelly wrote: > > I'd like to request some extra sets of eyes for the patch in ticket #10290. > > The summary: from r9838 on, I'm getting aggregation_regress test > failures in Oracle, apparently because annotations with extra selects > are adding the extra select aliases into the group by list. This > produces invalid SQL (at least in Oracle), since you can't use column > aliases defined at the same level within a group by clause. Sigh. You know, it's times like this that I'm really glad that we have a standard language that is common across all relational database implementations > The solution that I'm proposing is to use the extra select expression > itself in the group by, rather than the alias. This passes the tests > across all four included backends, and seems to work in general as > long as the expression is not a scalar subquery (in which case I think > a general solution would require advanced inspection of the > expression). But it bothers me a bit because it just feels fragile. We have to do a similar thing for the aggregates themselves; Postgres can't use an alias in the HAVING clause. However, Postgres doesn't seem to care about using aliases in the GROUP BY. In that case I erred on the side of caution and just made all backends use the full clause. If allowed alias usage is significantly different between backends (as it appears to be), it may be worth refactoring this. Out of interest - where does Oracle fall on this one? Can you use aliases in a HAVING or ORDER BY? > If there are no concerns or potential gotchas with this approach, then > I'll go ahead and commit this in a few days. My only major concern would be any potential performance hit from double-computing the extra clause. Alex's suggestion to parameterize this in the connection ops backend sounds like a good idea to me. However, rather than just defining a boolean feature flag, my preference would be to define a 'expand_extra()' function (or similar - bikeshed builder gets to choose the colour) on the ops backend which just returns the alias in the default case, but returns the full expression under Oracle. I should probably do the same for alias usage in HAVING/ORDER_BY clauses. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #10290: grouping with extra selects produces invalid SQL
On Tue, Feb 17, 2009 at 6:05 PM, Ian Kelly wrote: > > I'd like to request some extra sets of eyes for the patch in ticket #10290. > > The summary: from r9838 on, I'm getting aggregation_regress test > failures in Oracle, apparently because annotations with extra selects > are adding the extra select aliases into the group by list. This > produces invalid SQL (at least in Oracle), since you can't use column > aliases defined at the same level within a group by clause. > > The solution that I'm proposing is to use the extra select expression > itself in the group by, rather than the alias. This passes the tests > across all four included backends, and seems to work in general as > long as the expression is not a scalar subquery (in which case I think > a general solution would require advanced inspection of the > expression). But it bothers me a bit because it just feels fragile. > > If there are no concerns or potential gotchas with this approach, then > I'll go ahead and commit this in a few days. > > Thanks, > Ian > > > > As you mention it is fragile, and depending on how good the backends optimizer is it could require doing whatever computation 2x(which is undesirable), so I'd say make it an attribute on features and test for it when deciding which to use in group_by. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." --Voltaire "The people's good is the highest law."--Cicero --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
#10290: grouping with extra selects produces invalid SQL
I'd like to request some extra sets of eyes for the patch in ticket #10290. The summary: from r9838 on, I'm getting aggregation_regress test failures in Oracle, apparently because annotations with extra selects are adding the extra select aliases into the group by list. This produces invalid SQL (at least in Oracle), since you can't use column aliases defined at the same level within a group by clause. The solution that I'm proposing is to use the extra select expression itself in the group by, rather than the alias. This passes the tests across all four included backends, and seems to work in general as long as the expression is not a scalar subquery (in which case I think a general solution would require advanced inspection of the expression). But it bothers me a bit because it just feels fragile. If there are no concerns or potential gotchas with this approach, then I'll go ahead and commit this in a few days. Thanks, Ian --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 3:00 PM, Jeremy Dunck wrote: > > On Tue, Feb 17, 2009 at 3:52 PM, James Bennett wrote: >> >> On Tue, Feb 17, 2009 at 7:40 AM, NitinHayaran wrote: >>> Today i read this article and was wondering whether django orm is >>> really that bad. >>> >>> http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html >> >> Well, it's obligatory for me first to say "wow, Blogger sucks", since >> I can't actually read that post -- I just get a Blogger template with >> a big white empty space where the article ought to be (looking even at >> the HTML source, the content just ain't there). > > It used to be there. I think the OP deleted the post. I'm not sure. If you click the blog archive link for February, you can still read the full post. It's only the direct link that's not working, which means that the comments are (AFAIK) inaccessible. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 3:52 PM, James Bennett wrote: > > On Tue, Feb 17, 2009 at 7:40 AM, NitinHayaran wrote: >> Today i read this article and was wondering whether django orm is >> really that bad. >> >> http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html > > Well, it's obligatory for me first to say "wow, Blogger sucks", since > I can't actually read that post -- I just get a Blogger template with > a big white empty space where the article ought to be (looking even at > the HTML source, the content just ain't there). It used to be there. I think the OP deleted the post. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 7:40 AM, NitinHayaran wrote: > Today i read this article and was wondering whether django orm is > really that bad. > > http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html Well, it's obligatory for me first to say "wow, Blogger sucks", since I can't actually read that post -- I just get a Blogger template with a big white empty space where the article ought to be (looking even at the HTML source, the content just ain't there). -- "Bureaucrat Conrad, you are technically correct -- the best kind of correct." --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 3:15 PM, Jeremy Dunck wrote:
...
> {% for question in poll.questions.all.iterator %}
> works just fine.
>
OK, last one from me.
As a 2.0 wish, I'd like to make .iterator the default behavior, and
the cached-version a special case. I realize this point is quite
debatable.
However-- is there already a place for 2.0-wishlist sort of things? I
know there's no sense discussing in for the 1.x line. So how do we
remember these sorts of issues when it comes 2.x time?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 2:11 PM, Alex Gaynor wrote:
...
>
> Neither is completely correct ;). Both do chunked reads from the
> DB(__iter__ using iterator for getting the data), however __iter__ also
> caches them, so if you reiterate you don't do a second db query, whereas
> iterator doesn't cache them.
If I'm reading it right, it looks like ForNode doesn't use .iterator.
I can see why it might be useful to assume QS cache should be used--
maybe the same QS will be repeatedly iterated.
Even so, it seems like it'd be useful to have a built-in filter which
uses iter(object)?
{% for question in poll.questions.all()|iterate %}
?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 3:12 PM, Jeremy Dunck wrote:
> Even so, it seems like it'd be useful to have a built-in filter which
> uses iter(object)?
>
> {% for question in poll.questions.all()|iterate %}
Ugh.
Sorry, I'm an idiot.
{% for question in poll.questions.all.iterator %}
works just fine.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 1:50 PM, Ian Kelly wrote: > > On Tue, Feb 17, 2009 at 7:40 AM, M N Islam Shihan > wrote: > > > > Hi, > > > > Please go through the comments of the blog post you are referring and > you'll > > understand why, how and where to use the django ORM. > > > > Regards, > > Shihan > > Something I've just noticed here. One of the comments links to the > documentation for QuerySet.iterator: > http://docs.djangoproject.com/en/dev/ref/models/querysets/#iterator > > The documentation includes the statement, "A QuerySet typically reads > all of its results and instantiates all of the corresponding objects > the first time you access it; iterator() will instead read results and > instantiate objects in discrete chunks, yielding them one at a time." > > Am I mistaken, or is this not exactly correct? As I understand it, > the difference between QuerySet.__iter__ and QuerySet.iterator isn't > that the former reads and instantiates everything all at once, but > that the former will make use of the QuerySet's result cache, reading > from it when available and filling it as a side effect of iteration. > > Ian > > > > Neither is completely correct ;). Both do chunked reads from the DB(__iter__ using iterator for getting the data), however __iter__ also caches them, so if you reiterate you don't do a second db query, whereas iterator doesn't cache them. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." --Voltaire "The people's good is the highest law."--Cicero --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 7:40 AM, M N Islam Shihan wrote: > > Hi, > > Please go through the comments of the blog post you are referring and you'll > understand why, how and where to use the django ORM. > > Regards, > Shihan Something I've just noticed here. One of the comments links to the documentation for QuerySet.iterator: http://docs.djangoproject.com/en/dev/ref/models/querysets/#iterator The documentation includes the statement, "A QuerySet typically reads all of its results and instantiates all of the corresponding objects the first time you access it; iterator() will instead read results and instantiate objects in discrete chunks, yielding them one at a time." Am I mistaken, or is this not exactly correct? As I understand it, the difference between QuerySet.__iter__ and QuerySet.iterator isn't that the former reads and instantiates everything all at once, but that the former will make use of the QuerySet's result cache, reading from it when available and filling it as a side effect of iteration. Ian --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
Hi, Please go through the comments of the blog post you are referring and you'll understand why, how and where to use the django ORM. Regards, Shihan - Original Message - From: "NitinHayaran" To: "Django developers" Sent: Tuesday, February 17, 2009 6:40 PM Subject: Is this true. that django really takes a lot of memory? > > Hi All, > Today i read this article and was wondering whether django orm is > really that bad. > > http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html > > I think this is the right place to ask? > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
Hi, I did a reply on the post. Might be some time before it's approved. The gist is that yes it's that bad if you're using it naively. As long as you know what's gonna be loaded from the DB you can avoid those cases pretty easily. You can also check http://github.com/dcramer/django-idmapper/tree/master which is a rewrite of the infamous ticket 17 (http://code.djangoproject.com/ticket/17). Regards, Philippe On Tue, Feb 17, 2009 at 1:40 PM, NitinHayaran wrote: > > Hi All, > Today i read this article and was wondering whether django orm is > really that bad. > > http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html > > I think this is the right place to ask? > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is this true. that django really takes a lot of memory?
On Tue, Feb 17, 2009 at 6:40 AM, NitinHayaran wrote: > > Hi All, > Today i read this article and was wondering whether django orm is > really that bad. > > http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html > > I think this is the right place to ask? This seems to be about using Django rather than developing Django. I think django-users is the right place to ask rather than django-developers. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Is this true. that django really takes a lot of memory?
Hi All, Today i read this article and was wondering whether django orm is really that bad. http://dayhacker.blogspot.com/2009/02/why-django-orm-sucks-it-takes-hell-lot.html I think this is the right place to ask? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: App Engine port
No comments on the more efficient perms/contenttypes implementation? I'd really like to know if that is an option because if it's not I should better make app-engine-patch 1.0 compatible with the less efficient implementation. Hmm, or I could emulate those db tables in the backend, but that wouldn't be very nice. Bye, Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
