We yesterday encountered a program that in a degenerate case issued in a single transaction a huge number of selects (in a single transaction but each select in a separate call to PGExec) (huge = ~ 400,000).
That transaction would continue to eat memory up until a point where calls to malloc (in aset.c) would fail and log for example: ,"out of memory","Failed on request of size 11." Both from that transaction and random others. In additional observation of interest to us was that while both VIRT and RES was growing VIRT was always roughly double of RES. Which seems to indicate that whatever allocations were done not all of the memory allocated was actually touched (server is a Centos6 box). So I have two questions: - Is that expected expected behaviour? The transaction was in READ_COMMITED mode, and my best guess is that this implies that some snapshot is taken before each subselect and all of them are only freed once the transaction is finished - Any recommendations on the use of overcommit? We had it disabled on the assumption that with overcommit the OOM killer might kill a random process and that it is better to instead have a process that is actually allocating fail (and on the additional assumption that any memory allocated by postgres would actually be touched). This is not a huge production issue for us as we can fix the program to no longer issue huge numbers of selects. But we would like to understand. Thank you very much, Bene PS: The machine has huge amounts of memory and during normal operations it looks like this: -bash-4.1$ cat /proc/meminfo MemTotal: 49413544 kB MemFree: 1547604 kB Buffers: 5808 kB Cached: 43777988 kB SwapCached: 0 kB Active: 18794732 kB Inactive: 27309980 kB Active(anon): 13786796 kB Inactive(anon): 1411296 kB Active(file): 5007936 kB Inactive(file): 25898684 kB Unevictable: 71928 kB Mlocked: 55576 kB SwapTotal: 2047992 kB SwapFree: 2047992 kB Dirty: 12100 kB Writeback: 79684 kB AnonPages: 2393372 kB Mapped: 12887392 kB Shmem: 12871676 kB Slab: 1050468 kB SReclaimable: 190068 kB SUnreclaim: 860400 kB KernelStack: 4832 kB PageTables: 450584 kB NFS_Unstable: 23312 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 26754764 kB Committed_AS: 17394312 kB VmallocTotal: 34359738367 kB VmallocUsed: 120472 kB VmallocChunk: 34333956900 kB HardwareCorrupted: 0 kB AnonHugePages: 1599488 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 5604 kB DirectMap2M: 2078720 kB DirectMap1G: 48234496 kB -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers