Re: django 1.0.2, django_pagination-1.0.5, postgresql, LIMIT & OFFSET question

[Bartek SQ9MEV]

Karen Tracey pisze:
> On Wed, Feb 4, 2009 at 3:01 AM, Bartek SQ9MEV  > wrote:
[...]
> So, any idea where's the clue?
> Is it possible to use django-pagination with generic views without so
> senseless overhead?
> 
The cause was the {% if posts_list %} tag, which evaluates the queryset
before the limit was applied. Now it's obvious for me ;).
Django-pagination works well.
-- 
Bartek


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: django 1.0.2, django_pagination-1.0.5, postgresql, LIMIT & OFFSET question

[Karen Tracey]

On Wed, Feb 4, 2009 at 3:01 AM, Bartek SQ9MEV  wrote:

>
> Bartek pisze:
>
> > so... looks like everytime I need 20 Post items my database is hit by
> > query returning all records... I find it a big overhead, and my app
> > suffers from that...
> Today I'm still investigating this issue, and I've just tried to use
> Paginator object as described at
> http://docs.djangoproject.com/en/dev/topics/pagination/#topics-pagination
> with my Post model described ina my previous post. Funny thing is that
> it's OK -


Not so funny -- this is what I'd expect.  This is one of the main reasons
for using pagination, to limit the query results to a manageable size.


> sql for any page is:
>
> SELECT "galgather_post"."id", "galgather_post"."group_id",
> "galgather_post"."messageid", "galgather_post"."subject",
> "galgather_post"."author_id", "galgather_post"."posting_date",
> "galgather_post"."date_added" FROM "galgather_post" ORDER BY
> "galgather_post"."posting_date" DESC LIMIT 10 OFFSET 10
>
> triggered by:
> posts=Post.objects.all()
> p=Paginator(posts,10)
> page2=p.page(2)
> page2.object_list
>
> So, any idea where's the clue?
> Is it possible to use django-pagination with generic views without so
> senseless overhead?
>

You've got one test using base Django pagination routines that works
correctly, one using django-pagination that apparently issues an
unrestricted query, so it sounds like there's a problem in
django-pagination.  I assume you are using the latest of that?  I'd open an
issue on its project page.  You could also try to track down where in
django-pagination the unrestricted query is coming from and provide a
fix

Karen

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: django 1.0.2, django_pagination-1.0.5, postgresql, LIMIT & OFFSET question

[Bartek SQ9MEV]

Bartek pisze:

> so... looks like everytime I need 20 Post items my database is hit by
> query returning all records... I find it a big overhead, and my app
> suffers from that...
Today I'm still investigating this issue, and I've just tried to use
Paginator object as described at
http://docs.djangoproject.com/en/dev/topics/pagination/#topics-pagination
with my Post model described ina my previous post. Funny thing is that
it's OK - sql for any page is:

SELECT "galgather_post"."id", "galgather_post"."group_id",
"galgather_post"."messageid", "galgather_post"."subject",
"galgather_post"."author_id", "galgather_post"."posting_date",
"galgather_post"."date_added" FROM "galgather_post" ORDER BY
"galgather_post"."posting_date" DESC LIMIT 10 OFFSET 10

triggered by:
posts=Post.objects.all()
p=Paginator(posts,10)
page2=p.page(2)
page2.object_list

So, any idea where's the clue?
Is it possible to use django-pagination with generic views without so
senseless overhead?


-- 
Bartek


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

django 1.0.2, django_pagination-1.0.5, postgresql, LIMIT & OFFSET question

[Bartek]

Hi all
Using django-pagination, I encounter some serious performance problems
related to django-pagination. Request for every page triggers sql query
returning all Post instances, here's  detailed description of my problem:

My model Is:
class Post(models.Model):
group   = models.ForeignKey(Group, verbose_name='Grupa',)
messageid   = models.CharField("Message-Id", max_length=512)
subject = models.CharField("Subject", max_length=512)
author  = models.ForeignKey(Author, verbose_name="Autor")
posting_date= models.DateTimeField("NNTP-Posting-Date",
blank=True)
date_added  = models.DateTimeField("Dodany", auto_now_add=True)

def __unicode__(self):
return "[%s] %s" % (self.group, self.messageid)
class Meta:
ordering=['-posting_date']


and urls:
posts_dict={
'queryset': Post.objects.all(),
'template_name': 'galgather/posts_archive.html',
'allow_empty':True,
'template_object_name':'posts',
}
[...]
url(r'/$', object_list, dict(posts_dict), 'posts_archive',),
[...]

I want to paginate over Posts.objects.all(), so in my template i use:
{% autopaginate posts_list 20  %}
{% paginate %}
{% for post in posts_list %}
[...]
{% endfor %}

The problem is that every request for each page generates sql queries
fetching all objects, than django asks about Post count.
Queries looks like that:

SELECT "galgather_post"."id", "galgather_post"."group_id",
"galgather_post"."messageid", "galgather_post"."subject",
"galgather_post"."author_id", "galgather_post"."posting_date",
"galgather_post"."date_added" FROM "galgather_post" ORDER BY
"galgather_post"."posting_date" DESC

SELECT COUNT(*) FROM "galgather_post"


>>> Post.objects.count()
18145L

so... looks like everytime I need 20 Post items my database is hit by
query returning all records... I find it a big overhead, and my app
suffers from that...

I've read:
http://code.google.com/p/django-pagination/issues/detail?id=15
and
http://code.google.com/p/django-pagination/source/detail?r=27
and wandering if its normal behaviour? If not, what could the cause of
that nasty problem?

My python knowledge however still improoving, but still isn't sufficient
to quick analyze django code.

Thanks in advance

-- 
Bartek
spamtrap: http://sq9mev.info/spamtrap.html



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---