To start with I'd look at upgrading the project to the latest version of Django and Python as Python 3.10 and 3.11 have some performance improvements and you'll need to be on a Django 4 version to support them.
That aside it sounds like you've got good caching architecture setup since it's working well when cached but that the architecture without the caching is not up to the job. You need to investigate where the performance hit is coming in, most likely it'll be database performance. Once thing to check would be whether you are using things like like prefetch_related, select_related, and defer with any queryset or are you always returning the whole object. If you are returning the whole object do you need to? After that it depends on how you are using and exposing Django as to what I'd suggest you check next. Are you using DRF as API backend to the pages in S3 or are you serving pages via Django and then caching those? If it's the latter, run it locally with caching turned off and debug toolbar installed and look at each pages output to see how many queries you are hitting per page and the performance of each. Since you are using a lot of caching how are you maintaining the cache? Are you letting it expire and then self populate or are you using warm-ups to fetch the update information and then update the cache prior to it expiring? If you're doing the former but the latter fits your setup look to switch to that. In those instances you can update the fetch the results to update the cache outside of a user transaction On Monday, 12 June 2023 at 06:23:06 UTC+1 Kunal Mittal wrote: > Hi, we are using Django 1.8.5 > Architecture is like - Nginx side car -> wsgi server (gunicorn) -> > application > On Monday, June 12, 2023 at 1:06:18 AM UTC+5:30 Mz Gz wrote: > >> What server do you use ? >> >> On Sun, 11 Jun 2023, 10:00 pm Kunal Mittal, <mittal...@gmail.com> wrote: >> >>> I've a django application serving 10k+ users daily, that application has >>> everything to be on the top of performance metrics but still some API's >>> have high latencies, application serves lots of pages with nice css and >>> various images, on most the pages they are being served by s3 while on some >>> it's via django static files. We have implemented caching on akamai for >>> every static file that application serves, whenever that cache disappears >>> latency of each and every API spikes exponentially, at that time even >>> loading metrics becomes a pain, server does not pick those requests. Can >>> anyone suggest what's going on here. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-users...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/d32868ba-a2df-41e3-8f41-3ecc3c1b3471n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/django-users/d32868ba-a2df-41e3-8f41-3ecc3c1b3471n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/c5e4d83c-1891-4005-939e-8c0f3f074cf2n%40googlegroups.com.
