Re: Django "freezing" after a few minutes?
Found it! I'm using eventlet in some other code and I imported one of my modules into a django model. So eventlet was taking over and putting everything to "sleep". On Aug 20, 1:34 am, Thomas Guettler <h...@tbz-pariv.de> wrote: > erikcw schrieb: > > > Hi, > > > I'm running Django through mod_wsgi and Apache (2.2.8) on Ubuntu 8.04. > > > I've been running Django on this setup for about 6 months without any > > problems. Yesterday, I moved my database (postgres 8.3) to its own > > server, and my Django site started refusing to load (the browser > > spinner would just keep spinning). > > > Pages will load when I first start apache, but after about 10 mintues, > > it just stops. Apache is still able to serve static files. Just > > nothing through Django. > > The select timeout could be a network problem. Or a problem between > mod_wsgi and apache. > > Since it works during the first minutes, it does not look like > a network problem. Except you have a firewall that slows down > new connections if there are too many. > > Maybe you can find out with "ls -l /proc/PID/fd" and/or netstat/lsof > which tcp/ip connections the process waits for in the system call select. > > You can write a simple script with python and psycopg2 (without django, > just execute a simple SELECT statement) and check if you can still connect > after the pages don't get through django anymore. > > Thomas > > -- > Thomas Guettler,http://www.thomas-guettler.de/ > E-Mail: guettli (*) thomas-guettler + de --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Django "freezing" after a few minutes?
Hi Thomas,
Thanks for the reply.
I have postgres max_connections=100, but when I run "select count(*)
from pg_stat_activity;" I only get 2. So I would think that I'm not
maxing out the connections. (is that right -- I'm fairly new to
Postgres)
Here are the results from an strace:
# pgrep apache2
32555
32557
32561
32564
32565
32566
32568
32570
32600
32601
32630
32632
32633
r...@domu-12-31-39-03-3e-22:~# strace -p 32555
Process 32555 attached - interrupt to quit
select(0, NULL, NULL, NULL, {0, 48}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
waitpid(-1, 0xbf879bd8, WNOHANG|WSTOPPED) = 0
select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
Does any of this help?
Erik
On Aug 19, 8:09 am, Thomas Guettler <h...@tbz-pariv.de> wrote:
> Just a first guess:
>
> How many wsgi-clients access the db server at the same time?
> Maybe the db does accept only N, while apache/wsgi tries it
> with N+1. Then the connection could hang.
>
> What do the wsgi processes do? You can use "strace -p WSGI-PID" to find
> this out.
>
> You can sent SIGINT (like ctrl-c) the the PID. In the stacktrace
> you can see where the python code was hanging (should be in apache
> error log).
>
> HTH,
> Thomas
>
> erikcw schrieb:
>
>
>
> > Hi,
>
> > I'm running Django through mod_wsgi and Apache (2.2.8) on Ubuntu 8.04.
>
> > I've been running Django on this setup for about 6 months without any
> > problems. Yesterday, I moved my database (postgres 8.3) to its own
> > server, and my Django site started refusing to load (the browser
> > spinner would just keep spinning).
>
> > Pages will load when I first start apache, but after about 10 mintues,
> > it just stops. Apache is still able to serve static files. Just
> > nothing through Django.
>
> > I've checked the apache error logs, and I don't see any entries that
> > could be related. I'm not sure if this is a WSGI, Django, Apache, or
> > Postgres issue?
>
> --
> Thomas Guettler,http://www.thomas-guettler.de/
> E-Mail: guettli (*) thomas-guettler + de
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Django "freezing" after a few minutes?
Hi, I'm running Django through mod_wsgi and Apache (2.2.8) on Ubuntu 8.04. I've been running Django on this setup for about 6 months without any problems. Yesterday, I moved my database (postgres 8.3) to its own server, and my Django site started refusing to load (the browser spinner would just keep spinning). Pages will load when I first start apache, but after about 10 mintues, it just stops. Apache is still able to serve static files. Just nothing through Django. I've checked the apache error logs, and I don't see any entries that could be related. I'm not sure if this is a WSGI, Django, Apache, or Postgres issue? Any ideas? Thanks for your help! Erik --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Reducing the number of my view's database queries
Thanks for all of the responses. Just wanted to share where I'm at at
this point:
self.profilekeyword_set.select_related().all().extra(select={
'rank_url': 'SELECT url FROM rankreport_keywordrank WHERE
rankreport_keywordrank.keyword_id =
rankreport_profilekeyword.keyword_id AND
rankreport_keywordrank.site_id = rankreport_profile.site_id ORDER
BY `created` DESC LIMIT 0,1','competition': 'SELECT
competition FROM rankreport_keywordrank WHERE
rankreport_keywordrank.keyword_id =
rankreport_profilekeyword.keyword_id AND
rankreport_keywordrank.site_id = rankreport_profile.sit
e_id ORDER BY `created` DESC LIMIT 0,1',
'rank': 'SELECT rank FROM rankreport_keywordrank WHERE
rankreport_keywordrank.keyword_id =
rankreport_profilekeyword.keyword_id AND
rankreport_keywordrank.site_id = rankreport_profile.site_id ORDER BY
`created` DESC LIMIT 0,1',
'start_rank': 'SELECT rank FROM rankreport_keywordrank WHERE
rankreport_keywordrank.keyword_id =
rankreport_profilekeyword.keyword_id AND
rankreport_keywordrank.site_id = rankreport_profile.site_id ORD
ER BY `created` ASC LIMIT 0,1',
'avg': 'SELECT search_volume_avg FROM
rankreport_keywordmetrics WHERE rankreport_keywordmetrics.keyword_id =
rankreport_profilekeyword.keyword_id ORDER BY `created` DESC LIMIT
0,1',
'cpc': 'SELECT cpc FROM rankreport_keywordmetrics WHERE
rankreport_keywordmetrics.keyword_id =
rankreport_profilekeyword.keyword_id ORDER BY `created` DESC LIMIT
0,1',
'zz': 'start_rank - rank',
}).select_related()
This seems to work. If you know of any performance tweaks I'd love to
hear about them (however it is incredibly fast compared to my original
solution).
The only problem is that "zz" throws a database error.
(OperationalError: (1054, "Unknown column 'start_rank' in 'field
list'")) - ant ideas how I can get "zz" working?
Thanks again for the assistance!
Erik
On Dec 8, 4:49 pm, Malcolm Tredinnick
wrote:
> On Mon, 2008-12-08 at 07:43 -0800, DavidA wrote:
> > If I undestand the problem correctly, in MySQL you could do this in
> > one query as:
>
> > select
> > m.*,
> > (select min(created) from model2 where id = m.model2_id) as
> > first_created,
> > (select max(created) from model2 where id = m.model2_id) as
> > last_created
> > from model1 m
> > ;
>
> That will give the correct answer, however it's a bit inefficient, since
> the inner queries are run once for each model id (and in the original
> post model2 and model3 were used, so it's worse than it looks here).
> Since that means 6000 rows, based on Eric's original numbers, it's worth
> looking deeper. Let me cut-and-paste the original description for
> reference:
>
> (behold the power of a fully-functional, non top-posting email client!)
>
> > > > objects = Model1.objects.filter(user=3).select_related() #about 6,000
> > > > objects
>
> > > > data = {}
> > > > for o in objects:
> > > > data[o.name] = [o.field1, o.field2]
> > > > data[o.name].append(o.field3.model2_set.all().latest('created'))
> > > > #get latest row from related model2
> > > > data[o.name].append(o.model3_set.all().order_by('created')[0])
> > > > #get earliest row from related model3
>
> Ideally, there would be a way to run one inner query that does something
> like
>
> select max(created) from ... where model1.user_id = 3 group by
> model1.id
>
> except that isn't quite right because if the outer query only selected
> results with those created dates, it will get a superset of the right
> answer (some model2.id could have a created date that matches some other
> maximum but isn't the maximum for that particular model2.id, so you need
> to tie the maximum date to the model id).
>
> If something along those lines can be made to work, that inner query
> only has to run once (sure it's a big query, with a sort, but it's only
> one query and it's in the sweet spot of what databases can do). It's the
> difference between a correlated inner query (which you have -- so it
> runs once for each outer row) and an uncorrelated query like the above.
>
> Still, using the above query and then filtering out the dupes at the
> Python level might well be a good alternate solution. So you end up
> with:
>
> select model2.*, model1.id, ... from ...
> where model2.created in (select max(model2.created) from
> where model1.user_id=3 group by model1.id) order by
> model2.created asc
>
> Then you process these results, storing them in a dictionary keyed from
> model1.id. When you get duplicate results for the same model1.id (the
> above superset situation), the 'latest' result will be after the earlier
> ones and will overwrite them.
>
> The above is the query to select all the model2 fields. There's another
> query to select all the model3 fields. So that reduces the original
> 12001 queries to 3 and in the two with nested inner queries, the nested
> queries only run once each (or once per
Reducing the number of my view's database queries
Hi all,
I'm trying to write a model query that will return a queryset along
with the latest (and earliest) data from 2 related models.
Right now I'm doing something like this:
objects = Model1.objects.filter(user=3).select_related() #about 6,000
objects
data = {}
for o in objects:
data[o.name] = [o.field1, o.field2]
data[o.name].append(o.field3.model2_set.all().latest('created'))
#get latest row from related model2
data[o.name].append(o.model3_set.all().order_by('created')[0])
#get earliest row from related model3
The problem is that this results in a TON of database queries. This
view is taking over a minute to process. The select_related on the
first line doesn't seem to be helping since I'm using latest()/
order_by which generates a new query.
How can I make this more efficient? Denormalizing the isn't an option
since model2 and model 3 are many-to-one.
Is there something I can do with extra() in the first query or some
sort of subquey I can do to eliminate the loop?
Thanks!
Erik
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Named URLs for Flatpages?
Hi all,
I'm working on a project that uses flatpages pretty heavily. I was
wondering if there was a way to use named urls with flatpages so that
I can use reverse() in my other views and {% url flat_privacy_policy
%} in my templates.
I haven't been able to find anything helpful in the docs so far. Any
ideas?
Thanks!
Erik
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Audit Trail
Hi Sergio, I updated my code with your changes, and it still doesn't seem to be working. I don't have the history Manager in my model. Is it working for you? Are you using the latest trunk? Thanks! Erik On Aug 19, 12:15 pm, Sérgio Durand <[EMAIL PROTECTED]> wrote: > Hi people, > > Finally i've got Audit Trail [1] working again !!! :) > I'm not a django/python expert (i've started studying 2 months ago), but > after fighting with "python manage.py validate" and searching in google, > i've got the solution... the problem basically was missing **kwargs > arguments in some handler functions (because changes made in r8223). > Maybe this problem is very simple (in fact i think it is), but i've > spent several hours (and learned a lot of things too) to get this working. > This was my first code contribution and i'm very glad for that!! > > Sergio Durand > > [1]:http://code.djangoproject.com/wiki/AuditTrail > > ps: i hope i didn't make a code mistake and sorry my english !!! ;) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
