On Monday, February 9, 2015 at 1:50:58 PM UTC+5:30, Ryan Stuart wrote: > Hi Asad, > > Is there any reason why you can't just use profile/cProfile? In particular, > you could use the api of that module to save out the profile stats to an > external file with a unique name and then inspect them later with a tool like > snakeviz. The code to save profile stats might look like the following: > > > pr = cProfile.Profile() > pr.runcall(your_celery_task_without_async) > ps = pstats.Stats(pr) > ps.dump_stats("my_task.profile") > Obviously you need to call your celery task function directly, not via Celery > using delay() or any derivative. Alternatively, you could try something like > line_profiler by again, calling the task directly. > > > If using this method it turn out your actual task code is running quite fast, > then I'd suggest that the majority of the time is being lost in transferring > the task to the Celery node. > > Cheers > > > On Mon Feb 09 2015 at 5:20:43 AM Asad Dhamani <dhama...@gmail.com> wrote: > I have a Flask application where I run a specific task asynchronously using > Celery. Its basically parsing some HTML and inserting data into a Postgres > database(using SQLAlchemy). However, the task seems to be running very > slowly, at 1 insert per second. > > I'd like to find out where the bottleneck is, and I've been looking for a > good profiler that'd let me do this, however, I couldn't find anything. Any > recommendations would be great. > > -- > > https://mail.python.org/mailman/listinfo/python-list
Hi Ryan, I was looking for something that didn't make me modify my code. It seems like I'll just use cProfile with snakeviz. Thanks for the help -- https://mail.python.org/mailman/listinfo/python-list