On Fri, Apr 15, 2011 at 12:42 AM, Scott Carey wrote:
> I do know that dual-pivot quicksort provably causes fewer swaps (but the
> same # of compares) as the usual single-pivot quicksort. And swaps are a
> lot slower than you would expect due to the effects on processor caches.
> Therefore it migh
On 4/14/11 1:19 PM, "Claudio Freire" wrote:
>On Thu, Apr 14, 2011 at 10:05 PM, Scott Carey
>wrote:
>> Huge Pages helps caches.
>> Dual-Pivot quicksort is more cache friendly and is _always_ equal to or
>> faster than traditional quicksort (its a provably improved algorithm).
>
>If you want a ca
On Thu, Apr 14, 2011 at 10:05 PM, Scott Carey wrote:
> Huge Pages helps caches.
> Dual-Pivot quicksort is more cache friendly and is _always_ equal to or
> faster than traditional quicksort (its a provably improved algorithm).
If you want a cache-friendly sorting algorithm, you need mergesort.
I
On 4/13/11 9:23 PM, "Greg Smith" wrote:
>Scott Carey wrote:
>> If postgres is memory bandwidth constrained, what can be done to reduce
>> its bandwidth use?
>>
>> Huge Pages could help some, by reducing page table lookups and making
>> overall access more efficient.
>> Compressed pages (speedy
2011/4/14 Florian Weimer :
> * Jesper Krogh:
>
>> If you have a 1 socket system, all of your data can be fetched from
>> "local" ram seen from you cpu, on a 2 socket, 50% of your accesses
>> will be "way slower", 4 socket even worse.
>
> There are non-NUMA multi-socket systems, so this doesn't appl
* Jesper Krogh:
> If you have a 1 socket system, all of your data can be fetched from
> "local" ram seen from you cpu, on a 2 socket, 50% of your accesses
> will be "way slower", 4 socket even worse.
There are non-NUMA multi-socket systems, so this doesn't apply in all
cases. (The E5320-based sy
Scott Carey wrote:
If postgres is memory bandwidth constrained, what can be done to reduce
its bandwidth use?
Huge Pages could help some, by reducing page table lookups and making
overall access more efficient.
Compressed pages (speedy / lzo) in memory can help trade CPU cycles for
memory usage
If postgres is memory bandwidth constrained, what can be done to reduce
its bandwidth use?
Huge Pages could help some, by reducing page table lookups and making
overall access more efficient.
Compressed pages (speedy / lzo) in memory can help trade CPU cycles for
memory usage for certain memory se
--- On Tue, 12/4/11, Greg Smith wrote:
> From: Greg Smith
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: "Kevin Grittner"
> Cc: da...@lang.hm, "Steve Clark" , "Glyn Astill"
> , "Joshua D. Drake" , "Scott
>
ng.hm; Steve Clark; Glyn Astill; Joshua D. Drake;
Scott Marlowe; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith wrote:
> Kevin Grittner wrote:
>>
>> Glyn Astill wrote:
>>
>&g
On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith wrote:
> Kevin Grittner wrote:
>>
>> Glyn Astill wrote:
>>
>>>
>>> Results from Greg Smiths stream_scaling test are here:
>>>
>>> http://www.privatepaste.com/4338aa1196
>>>
>>
>> Well, that pretty much clinches it. Your RAM access tops out at 16
>> p
Scott Marlowe wrote:
Have you tried running the memory stream benchmark Greg Smith had
posted here a while back? It'll let you know if you're memory is
bottlenecking. Right now my 48 core machines are the king of that
benchmark with something like 70+Gig a second.
The big Opterons are stil
Kevin Grittner wrote:
Glyn Astill wrote:
Results from Greg Smiths stream_scaling test are here:
http://www.privatepaste.com/4338aa1196
Well, that pretty much clinches it. Your RAM access tops out at 16
processors. It appears that your processors are spending most of
their time w
Hi,
I think that a NUMA architecture machine can solve the problem
A +
Le 11/04/2011 15:04, Glyn Astill a écrit :
Hi Guys,
I'm just doing some tests on a new server running one of our heavy select
functions (the select part of a plpgsql function to allocate seats)
concurrently. We do u
On Tue, Apr 12, 2011 at 6:40 PM, Kevin Grittner
wrote:
>
> Well, that pretty much clinches it. Your RAM access tops out at 16
> processors. It appears that your processors are spending most of
> their time waiting for and contending for the RAM bus.
It tops, but it doesn't drop.
I'd propose th
Glyn Astill wrote:
> Results from Greg Smiths stream_scaling test are here:
>
> http://www.privatepaste.com/4338aa1196
Well, that pretty much clinches it. Your RAM access tops out at 16
processors. It appears that your processors are spending most of
their time waiting for and contending fo
On Tue, Apr 12, 2011 at 11:01 AM, Glyn Astill wrote:
> --- On Tue, 12/4/11, Merlin Moncure wrote:
>
>> >>> Can we see some iobound and cpubound pgbench
>> runs on both
>> >>> servers?
>> >>>
>> >>
>> >> Of course, I'll post when I've gotten to that.
>> >
>> > Ok, there's no writing going on -- so
--- On Tue, 12/4/11, Kevin Grittner wrote:
> Wow, zero idle and zero wait, and single digit for
> system. Did you
> ever run those RAM speed tests? (I don't remember
> seeing results
> for that -- or failed to recognize them.) At this
> point, my best
> guess at this point is that you don't ha
--- On Tue, 12/4/11, Merlin Moncure wrote:
> >>> Can we see some iobound and cpubound pgbench
> runs on both
> >>> servers?
> >>>
> >>
> >> Of course, I'll post when I've gotten to that.
> >
> > Ok, there's no writing going on -- so the i/o tets
> aren't necessary.
> > Context switches are also n
Glyn Astill wrote:
> Tried tweeking LOG2_NUM_LOCK_PARTITIONS between 5 and 7. My
> results took a dive when I changed to 32 partitions, and improved
> as I increaced to 128, but appeared to be happiest at the default
> of 16.
Good to know.
>> Also, if you can profile PostgreSQL at the sweet
On Tue, Apr 12, 2011 at 8:23 AM, Merlin Moncure wrote:
> On Tue, Apr 12, 2011 at 3:54 AM, Glyn Astill wrote:
>> --- On Tue, 12/4/11, Merlin Moncure wrote:
>>
>>> >> The issue I'm seeing is that 8 real cores
>>> outperform 16 real
>>> >> cores, which outperform 32 real cores under high
>>> concur
On Tue, Apr 12, 2011 at 3:54 AM, Glyn Astill wrote:
> --- On Tue, 12/4/11, Merlin Moncure wrote:
>
>> >> The issue I'm seeing is that 8 real cores
>> outperform 16 real
>> >> cores, which outperform 32 real cores under high
>> concurrency.
>> >
>> > With every benchmark I've done of PostgreSQL, t
--- On Mon, 11/4/11, Kevin Grittner wrote:
> From: Kevin Grittner
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: da...@lang.hm, "Steve Clark" , "Kevin Grittner"
> , "Glyn Astill"
> Cc: "Joshua D. Drake" , "
--- On Tue, 12/4/11, Scott Marlowe wrote:
> From: Scott Marlowe
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: "Glyn Astill"
> Cc: pgsql-performance@postgresql.org
> Date: Tuesday, 12 April, 2011, 6:55
> On Mon, Apr 11, 2011 at 7:04 AM, Glyn
&g
--- On Tue, 12/4/11, Merlin Moncure wrote:
> >> The issue I'm seeing is that 8 real cores
> outperform 16 real
> >> cores, which outperform 32 real cores under high
> concurrency.
> >
> > With every benchmark I've done of PostgreSQL, the
> "knee" in the
> > performance graph comes right around ((
On 11-4-2011 22:04 da...@lang.hm wrote:
in your case, try your new servers without hyperthreading. you will end
up with a 4x4 core system, which should handily outperform the 2x4 core
system you are replacing.
the limit isn't 8 cores, it's that the hyperthreaded cores don't work
well with the p
On Mon, Apr 11, 2011 at 7:04 AM, Glyn Astill wrote:
> Hi Guys,
>
> I'm just doing some tests on a new server running one of our heavy select
> functions (the select part of a plpgsql function to allocate seats)
> concurrently. We do use connection pooling and split out some selects to
> slony
On 2011-04-11 22:39, James Cloos wrote:
"GA" == Glyn Astill writes:
GA> I was hoping someone had seen this sort of behaviour before,
GA> and could offer some sort of explanation or advice.
Jesper's reply is probably most on point as to the reason.
I know that recent Opterons use some of the
On Mon, Apr 11, 2011 at 6:50 PM, mark wrote:
>
> Interesting, thanks for sharing.
>
> I guess I have never gotten to the point where I felt I needed more than 2
> drives for my xlogs. Maybe I have been dismissing that as a possibility
> something. (my biggest array is only 24 SFF drives tho)
>
> I
On Mon, Apr 11, 2011 at 5:06 PM, Kevin Grittner
wrote:
> Glyn Astill wrote:
>
>> The issue I'm seeing is that 8 real cores outperform 16 real
>> cores, which outperform 32 real cores under high concurrency.
>
> With every benchmark I've done of PostgreSQL, the "knee" in the
> performance graph co
> -Original Message-
> From: Scott Marlowe [mailto:scott.marl...@gmail.com]
> Sent: Monday, April 11, 2011 6:18 PM
> To: mark
> Cc: Glyn Astill; Kevin Grittner; Joshua D. Drake; pgsql-
> performa...@postgresql.org
> Subject: Re: [PERFORM] Linux: more cores = less con
On Mon, Apr 11, 2011 at 6:18 PM, Scott Marlowe wrote:
> On Mon, Apr 11, 2011 at 6:05 PM, mark wrote:
>> Just wondering, which LSI card ?
>> Was this 32 drives in Raid 1+0 with a two drive raid 1 for logs or some
>> other config?
>
> We were using teh LSI but I'll be switching back to Areca wh
On Mon, Apr 11, 2011 at 6:05 PM, mark wrote:
> Just wondering, which LSI card ?
> Was this 32 drives in Raid 1+0 with a two drive raid 1 for logs or some
> other config?
We were using teh LSI but I'll be switching back to Areca when we
go back to HW RAID. The LSI only performed well if w
.org
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
>
> On Mon, Apr 11, 2011 at 12:23 PM, Glyn Astill
> wrote:
> >
> >
> > --- On Mon, 11/4/11, Joshua D. Drake wrote:
> >
> >> From: Joshua D. Drake
> >> Subject: Re: [PERFORM]
On Mon, Apr 11, 2011 at 6:04 AM, Glyn Astill wrote:
> The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz, our current servers
> are 2 x 4 core Xeon E5320 CPUs at 2Ghz.
>
> What I'm seeing is when the number of clients is greater than the number of
> cores, the new servers perform better on f
"Kevin Grittner" wrote:
> I don't know why you were hitting the knee sooner than I've seen
> in my benchmarks
If you're compiling your own executable, you might try boosting
LOG2_NUM_LOCK_PARTITIONS (defined in lwlocks.h) to 5 or 6. The
current value of 4 means that there are 16 partitions to
> "GA" == Glyn Astill writes:
GA> I was hoping someone had seen this sort of behaviour before,
GA> and could offer some sort of explanation or advice.
Jesper's reply is probably most on point as to the reason.
I know that recent Opterons use some of their cache to better manage
cache-cohere
--- On Mon, 11/4/11, Scott Marlowe wrote:
> From: Scott Marlowe
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: "Glyn Astill"
> Cc: "Kevin Grittner" , "Joshua D. Drake"
> , pgsql-performance@postgresql.org
> Date: Monday,
Glyn Astill wrote:
> The issue I'm seeing is that 8 real cores outperform 16 real
> cores, which outperform 32 real cores under high concurrency.
With every benchmark I've done of PostgreSQL, the "knee" in the
performance graph comes right around ((2 * cores) +
effective_spindle_count). With
On Mon, Apr 11, 2011 at 1:42 PM, Glyn Astill wrote:
> A wild guess is something like multiple cores contending for cpu cache, cpu
> affinity, or some kind of contention in the kernel, alas a little out of my
> depth.
>
> It's pretty sickening to think I can't get anything else out of more than
On Mon, 11 Apr 2011, Steve Clark wrote:
On 04/11/2011 02:32 PM, Scott Marlowe wrote:
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake
wrote:
On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner"
wrote:
Glyn Astill wrote:
The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
Which has h
--- On Mon, 11/4/11, da...@lang.hm wrote:
> From: da...@lang.hm
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: "Steve Clark"
> Cc: "Scott Marlowe" , "Joshua D. Drake"
> , "Kevin Grittner" ,
> pgsql-performan
On 2011-04-11 21:42, Glyn Astill wrote:
I'll have to try with the synthetic benchmarks next then, but somethings
definately going off here. I'm seeing no disk activity at all as they're
selects and all pages are in ram.
Well, if you dont have enough computations to be bottlenecked on the
cpu
On 04/11/2011 02:32 PM, Scott Marlowe wrote:
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake
wrote:
On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner"
wrote:
Glyn Astill wrote:
The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
Which has hyperthreading.
our current servers are
--- On Mon, 11/4/11, Scott Marlowe wrote:
> Just FYI, in synthetic pgbench type benchmarks, a 48 core
> AMD Magny
> Cours with LSI HW RAID and 34 15k6 Hard drives scales
> almost linearly
> up to 48 or so threads, getting into the 7000+ tps
> range. With SW
> RAID it gets into the 5500 tps range
On Mon, Apr 11, 2011 at 12:23 PM, Glyn Astill wrote:
>
>
> --- On Mon, 11/4/11, Joshua D. Drake wrote:
>
>> From: Joshua D. Drake
>> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
>> To: "Kevin Grittner"
>> Cc: pgsql-performance@post
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake
wrote:
> On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner"
> wrote:
>> Glyn Astill wrote:
>>
>>> The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
>>
>> Which has hyperthreading.
>>
>>> our current servers are 2 x 4 core Xeon E5320 CPUs
--- On Mon, 11/4/11, Joshua D. Drake wrote:
> From: Joshua D. Drake
> Subject: Re: [PERFORM] Linux: more cores = less concurrency.
> To: "Kevin Grittner"
> Cc: pgsql-performance@postgresql.org, "Glyn Astill"
> Date: Monday, 11 April, 2011, 19:12
&
On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner"
wrote:
> Glyn Astill wrote:
>
>> The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
>
> Which has hyperthreading.
>
>> our current servers are 2 x 4 core Xeon E5320 CPUs at 2Ghz.
>
> Which doesn't have hyperthreading.
>
> PostgreS
Glyn Astill wrote:
> The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
Which has hyperthreading.
> our current servers are 2 x 4 core Xeon E5320 CPUs at 2Ghz.
Which doesn't have hyperthreading.
PostgreSQL often performs worse with hyperthreading than without.
Have you turned HT off
Hi Guys,
I'm just doing some tests on a new server running one of our heavy select
functions (the select part of a plpgsql function to allocate seats)
concurrently. We do use connection pooling and split out some selects to slony
slaves, but the tests here are primeraly to test what an individ
51 matches
Mail list logo
