On 2018-Aug-06, Kohei KaiGai wrote: > 2018-08-06 1:50 GMT+09:00 Alvaro Herrera <alvhe...@2ndquadrant.com>: > >> Now, it consumed about 60MB rss at the beginning of COPY FROM, and it > >> grows up very slowly during the COPY FROM execution, then grew up to > >> 250MB before completion. > >> We may have another memory blocks which are not released during > >> execution, however, > >> I could not identify whether it is really memory leaking or not, and > >> who's jobs. > > > > Most likely, this is a different memory leak. > > > > I sugges that one way to track this down is first figure out *which* > > context is the one growing, which you can see by running > > MemoryContextStats a few times and noting for meaningful differences. > > Then we can try to narrow down what is allocating stuff in that context. > > > Yes, but the hardest is identification of which memory context is growing > up time by time. Once we could identify, MemoryContextStats() tells us > the name of problematic context and details.
Well, I was thinking you'd call MemCxtStats on TopMemoryContext and observe changes in the whole hierarchy. However ... > Of course, above my observation is just based on rss usage of postgresql. > It can increase physical page allocation by page fault on the virtual address > space correctly allocated. ... this is a good point too, and I'm not sure to what extent this problem is fixable. > >> It may be an idea to put a debug code that raises a notice when > >> MemoryContext allocates more than the threshold. > > > > I don't think this is really practical, because whatever the threshold > > we set, there would be some corner-case scenario where the threshold is > > legitimately crossed. And some memory leak scenarios that don't cross > > any thresholds. > > > I assume this threshold is configurable by GUC, and disabled on the default. > Once a user found suspicious memory usage increase, we can set a threshold > value. In above case, we may be able to see something around 120MB threshold. Okay. I suppose you'd want to improve traceability of allocations in some more general way, but I think I understand your point about the threshold. Seems overly specific, but maybe it's okay. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services