On Tue, Aug 30, 2011 at 10:05 PM, Scott Marlowe scott.marl...@gmail.com wrote:
On Tue, Aug 30, 2011 at 8:36 PM, mark dvlh...@gmail.com wrote:
To the broader list, regarding troubles with kswap. I am curious to what
others seeing from /proc/zoneinfo for DMA pages (not dma32 or normal) -
On Tue, Aug 30, 2011 at 9:05 PM, Scott Marlowe scott.marl...@gmail.com wrote:
Well, we had zone reclaim mode autoset to 1, and we had to turn it off
to get decent performance with postgresql. Machine was a quad
dodecacore Magny Cours, so 48 cores with 128G RAM. RAID controller is
an Areca
* Scott Marlowe:
On a machine with lots of memory, I've run into pathological behaviour
with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
eating up CPU and swap io like mad, while doing essentially nothing.
Setting swappiness to 0 delayed this behaviour but did not stop
On Tue, Aug 30, 2011 at 12:54 AM, Florian Weimer fwei...@bfk.de wrote:
* Scott Marlowe:
On a machine with lots of memory, I've run into pathological behaviour
with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
eating up CPU and swap io like mad, while doing essentially
On Tue, Aug 30, 2011 at 10:39 AM, Scott Marlowe scott.marl...@gmail.com wrote:
On Tue, Aug 30, 2011 at 12:54 AM, Florian Weimer fwei...@bfk.de wrote:
* Scott Marlowe:
On a machine with lots of memory, I've run into pathological behaviour
with both the RHEL 5 and Ubuntu 10.04 kernels where the
On a machine with lots of memory, I've run into pathological behaviour
with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
eating up CPU and swap io like mad, while doing essentially nothing.
Setting swappiness to 0 delayed this behaviour but did not stop
Yes, a few hundred MB of swap, and its definitely making a huge
difference. Upon restarting postgres, its all freed up, and then perf
is good again. Also, this box only has 1GB of swap total, so its
never going to get up a few dozen GB.
Anyway, here's some of top output
On Tue, Aug 30, 2011 at 2:50 AM, Sim Zacks s...@compulab.co.il wrote:
On a machine with lots of memory, I've run into pathological behaviour
with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
eating up CPU and swap io like mad, while doing essentially nothing.
Setting
On Tue, Aug 30, 2011 at 2:50 AM, Sim Zacks s...@compulab.co.il wrote:
Instead of restarting the database try swapoff -a swapon -a and see if
that helps performance. If it is that little swap in use, it might be
something else clogging up the works.
Check to see if kswapd is going crazy or
Hi,
2011-08-29 22:36 keltezéssel, Lonni J Friedman írta:
... I read that
(max_connections * work_mem) should never exceed physical RAM, and if
that's accurate, then I suspect that's the root of my problem on
systemA (below).
work_mem is process-local memory so
(max_connections * work_mem)
It is recommended to identify the processes using up high work_mem and try
to set work_mem to higher value at the session level.
I this case, all the connections using up maximum work_mem is the potential
threat. As said by Zoltan, work_mem is very high and shared_buffers as well.
Other
Lonni J Friedman wrote:
ok, I'll do my best to capture this data, and then reply back.
If using linux, you should find interesting data on per-process swap and
memory usage in /proc/${pid}/smaps
Also consider the script here:
On Tue, Aug 30, 2011 at 1:26 AM, Greg Smith g...@2ndquadrant.com wrote:
I doubt this has anything to do with your problem, just pointing this out as
future guidance. Until there's a breakthrough in the PostgreSQL buffer
cache code, there really is no reason to give more than 8GB of dedicated
On Mon, Aug 29, 2011 at 6:54 PM, peixubin peixu...@yahoo.com.cn wrote:
You should monitor PageTables value in /proc/meminfo.if the value larger than
1G,I Suggest enable hugepages .
To monitor PageTables:
# cat /proc/meminfo |grep -i pagetables
$ cat /proc/meminfo |grep -i pagetables
On Tue, Aug 30, 2011 at 3:00 AM, Boszormenyi Zoltan z...@cybertec.at wrote:
Hi,
2011-08-29 22:36 keltezéssel, Lonni J Friedman írta:
... I read that
(max_connections * work_mem) should never exceed physical RAM, and if
that's accurate, then I suspect that's the root of my problem on
On Mon, Aug 29, 2011 at 5:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Lonni J Friedman netll...@gmail.com writes:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have fairly substantial amounts of RAM
On Tue, Aug 30, 2011 at 11:17 AM, Lonni J Friedman netll...@gmail.com wrote:
On Mon, Aug 29, 2011 at 5:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Lonni J Friedman netll...@gmail.com writes:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap
Merlin Moncure mmonc...@gmail.com writes:
On Tue, Aug 30, 2011 at 11:17 AM, Lonni J Friedman netll...@gmail.com wrote:
In the past 18 hours, swap usage has nearly doubled on systemA:
$ free -m
total used free sharedbuffers cached
Mem: 56481
On 08/30/11 12:18 PM, Tom Lane wrote:
total used free sharedbuffers cached
Mem: 56481 55486995 0 15 53298
-/+ buffers/cache: 2172 54309
Swap: 1099 18 1081
This is totally
On Tue, Aug 30, 2011 at 2:55 PM, John R Pierce pie...@hogranch.com wrote:
On 08/30/11 12:18 PM, Tom Lane wrote:
total used free shared buffers cached
Mem: 56481 55486 995 0 15
53298
-/+ buffers/cache: 2172
On Tue, Aug 30, 2011 at 5:05 PM, Lonni J Friedman netll...@gmail.com wrote:
On Tue, Aug 30, 2011 at 2:55 PM, John R Pierce pie...@hogranch.com wrote:
On 08/30/11 12:18 PM, Tom Lane wrote:
total used free shared buffers cached
Mem: 56481 55486
-Original Message-
From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-
ow...@postgresql.org] On Behalf Of Scott Marlowe
Sent: Tuesday, August 30, 2011 3:52 AM
To: Sim Zacks
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] heavy swapping, not sure why
On Tue
On Tue, Aug 30, 2011 at 8:36 PM, mark dvlh...@gmail.com wrote:
Scott,
1000 max connections ? I thought that was several times more than
recommended these days, even for 24 or 48 core machines. Or am I living in
the past ? (I admit that my most recent runs of pgbench showed that best
On Tue, Aug 30, 2011 at 8:36 PM, mark dvlh...@gmail.com wrote:
To the broader list, regarding troubles with kswap. I am curious to what
others seeing from /proc/zoneinfo for DMA pages (not dma32 or normal) -
basically if it sits at 1 or not. Setting swappiness to 0 did not have any
affect for
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have fairly substantial amounts of RAM (not including swap), yet the
amount of swap that postgres is using ramps up over time and
eventually hurts performance
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time.
It's the Linux kernel that does it, not PostgreSQL. Set vm.swappiness=0
(usually in /etc/sysctl.conf) and put
On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time.
It's the Linux kernel that does it, not
On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman netll...@gmail.com wrote:
On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing
On Mon, Aug 29, 2011 at 3:46 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time.
It's the Linux kernel that does it, not
On Mon, Aug 29, 2011 at 3:36 PM, Lonni J Friedman netll...@gmail.com wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have fairly substantial amounts of RAM (not including swap), yet the
amount of
On Mon, Aug 29, 2011 at 2:24 PM, Scott Marlowe scott.marl...@gmail.com wrote:
On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman netll...@gmail.com wrote:
On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have
On Mon, Aug 29, 2011 at 2:38 PM, Merlin Moncure mmonc...@gmail.com wrote:
On Mon, Aug 29, 2011 at 3:36 PM, Lonni J Friedman netll...@gmail.com wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have
On Mon, Aug 29, 2011 at 3:38 PM, Lonni J Friedman netll...@gmail.com wrote:
On Mon, Aug 29, 2011 at 2:24 PM, Scott Marlowe scott.marl...@gmail.com
wrote:
On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman netll...@gmail.com wrote:
On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson
On Mon, Aug 29, 2011 at 2:51 PM, Scott Marlowe scott.marl...@gmail.com wrote:
On Mon, Aug 29, 2011 at 3:38 PM, Lonni J Friedman netll...@gmail.com wrote:
On Mon, Aug 29, 2011 at 2:24 PM, Scott Marlowe scott.marl...@gmail.com
wrote:
On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman
On Mon, Aug 29, 2011 at 4:45 PM, Lonni J Friedman netll...@gmail.com wrote:
using any C code in the backend? this includes 3rd party libraries
which link in C, including postgis, pljava, xml2, etc. Any features
being used not included in the standard core distribution are
interesting.
Nope,
On Mon, Aug 29, 2011 at 3:17 PM, Merlin Moncure mmonc...@gmail.com wrote:
On Mon, Aug 29, 2011 at 4:45 PM, Lonni J Friedman netll...@gmail.com wrote:
using any C code in the backend? this includes 3rd party libraries
which link in C, including postgis, pljava, xml2, etc. Any features
being
On August 29, 2011 02:34:26 PM you wrote:
On Mon, Aug 29, 2011 at 3:46 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over
Lonni J Friedman netll...@gmail.com writes:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have fairly substantial amounts of RAM (not including swap), yet the
amount of swap that postgres is using
On Mon, Aug 29, 2011 at 5:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Lonni J Friedman netll...@gmail.com writes:
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have fairly substantial amounts of RAM
You should monitor PageTables value in /proc/meminfo.if the value larger than
1G,I Suggest enable hugepages .
To monitor PageTables:
# cat /proc/meminfo |grep -i pagetables
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
On Mon, Aug 29, 2011 at 5:01 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 02:34:26 PM you wrote:
On Mon, Aug 29, 2011 at 3:46 PM, Alan Hodgson ahodg...@simkin.ca wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated
On 08/29/2011 06:12 PM, Lonni J Friedman wrote:
OK, I'll reduce it to 10GB and see if there's any noticable change in
performance. thanks
I've never heard a report of a Linux system using more than 8GB of
shared_buffers usefully, and peak performance on systems I've tested has
42 matches
Mail list logo
