Nevermind…select_related() did the trick. 226 queries cut to 4…amazing this little statement ☺
. . . . . . . . . . . . . . . . . . . . . . . . . . Ing. Patrick Szabo Developer LexisNexis A-1030 Wien, Marxergasse 25 patrick.sz...@lexisnexis.at<mailto:patrick.sz...@lexisnexis.at> Tel.: +43 1 53452 1573 Fax.: +43 1 534 52 146 . . . . . . . . . . . . . . . . . . . . . . . . . . Von: django-users@googlegroups.com [mailto:django-users@googlegroups.com] Im Auftrag von Szabo, Patrick (LNG-VIE) Gesendet: Dienstag, 18. Dezember 2012 08:08 An: django-users@googlegroups.com Betreff: AW: problem with timeout Thanks, that does sound like a better idea and now that i think about it i saw this behaviour on many other webinterfaces already…don’t know why i didn’t think about that ☺ However I’ve noticed that it’s not the query that is taking so long, it’s the processing of the result that takes ~180 seconds…yes 10 minutes might have been a little exaggerated ;-) It’s this code: for buchung in buchungen: if str(buchung.Produkt.Bestell_Nr): buch_pool[str(buchung.Produkt) + " #" + str(buchung.Produkt.Bestell_Nr)].append([str(buchung.Aktivitaet), str(buchung.Minuten)]) else: buch_pool[str(buchung.Produkt)].append([str(buchung.Aktivitaet), str(buchung.Minuten)]) You mightbe wondering why I’m even building such a weird construct. Truth is, this thing has grown over time with different devs working on it and I don’t have time to change it. My guess is that this code is triggering a load of queries…any idea how to make this more efficient ? Von: django-users@googlegroups.com<mailto:django-users@googlegroups.com> [mailto:django-users@googlegroups.com] Im Auftrag von Russell Keith-Magee Gesendet: Dienstag, 18. Dezember 2012 00:55 An: django-users@googlegroups.com<mailto:django-users@googlegroups.com> Betreff: Re: problem with timeout On Mon, Dec 17, 2012 at 7:34 PM, Szabo, Patrick (LNG-VIE) <patrick.sz...@lexisnexis.at<mailto:patrick.sz...@lexisnexis.at>> wrote: Hi, I have an operation that takes about 10 minutes to befinished. It takes that long because our DB is pretty big. This produces a timeout. I have tried so set the timeout in apache higher but it seems that apache is not taking this directive....just keeps timing out after 300 seconds. Is there anything I can to to overcome this within my app ? Yes. Stop doing long lived operations in a web request. Seriously, stop. :-) HTTP is designed to be a large number of short lived operations, not long lived operations. At the moment, you're hitting a resource limit at the HTTP server, but I'd be deeply surprised if that's the only problem you hit. HTTP just isn't designed to work in the way you've described. If you need to perform a long lived operation, you need to structure your workflow differently. Instead of a single request that takes 10 minutes to create a response, you: 1) Issue a request that creates a record in the database with all the details necessary to start the "job" 2) Return to the user a success message that indicates work has started. 3) In the background, in a *completely separate process*, process the job. 4) When the job is finished, insert the results into the database, and mark the job as finished 5) If the user hits the page representing the job, they can view the results. The important detail here is that you get the heavy lifting *out* of the request-response cycle. How you achieve that is up to you. On the very simple end, you just have a job status page that the user manually reloads, and a cron task in the background to do the heavy lifting. On the complex end, you could use long polling or web sockets to provide the status update, and something like Celery to handle the task management. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com<mailto:django-users@googlegroups.com>. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com<mailto:django-users+unsubscr...@googlegroups.com>. For more options, visit this group at http://groups.google.com/group/django-users?hl=en. . . . . . . . . . . . . . . . . . . . . . . . . . . Ing. Patrick Szabo Developer LexisNexis A-1030 Wien, Marxergasse 25 patrick.sz...@lexisnexis.at<mailto:patrick.sz...@lexisnexis.at> Tel.: +43 1 53452 1573 Fax.: +43 1 534 52 146 . . . . . . . . . . . . . . . . . . . . . . . . . . -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com<mailto:django-users@googlegroups.com>. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com<mailto:django-users+unsubscr...@googlegroups.com>. For more options, visit this group at http://groups.google.com/group/django-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
