On Sun, Dec 1, 2019 at 12:15 PM José María Mateos <ch...@rinzewind.org> wrote: > print("Memory usage:", psutil.Process(os.getpid()).memory_info().rss) > > Notice that memory usage increases noticeably specially on files 4 and > 5, the biggest ones, and doesn't come down as I would expect it to. But > the loading time is constant, so I think I can disregard any pickle > caching mechanisms. > > So I guess now my question is: can anyone give me any pointers as to why > is this happening? Any help is appreciated. >
I can't answer your question authoritatively, but I can suggest a place to look. Python's memory allocator doesn't always return memory to the system when the objects are freed up, for various reasons including the way that memory pages get allocated from. But it internally knows which parts are in use and which parts aren't. You're seeing the RSS go down slightly at some points, which would be the times when entire pages can be released; but other than that, what you'll end up with is a sort of high-water-mark with lots of unused space inside it. So what you're seeing isn't actual objects being cached, but just memory ready to be populated with future objects. ChrisA -- https://mail.python.org/mailman/listinfo/python-list