Re: 8.4 Performance improvements: was Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-13 Thread Greg Smith

On Fri, 13 Mar 2009, Jignesh K. Shah wrote:

I can use dbt2, dbt3 tests to see how 8.4 performs and compare it with 
8.3?


That would be very helpful.  There's been some work at updating the DTrace 
capabilities available too; you might compare what that's reporting too.


* Visibility map - Reduce Vacuum overhead - (I think I can time vacuum with 
some usage on both databases)


The reduced vacuum overhead should show up as just better overall 
performance.  If you can seperate out the vacuum specific time that would 
be great, I don't know that it's essential.  If the changes don't just 
make a plain old speed improvement in your tests that would be a problem 
worth reporting.



* Parallel pg_restore (Can be tested with a big database dump)


It would be particularly useful if you could throw some of your 32+ core 
systems at a parallel restore of something with a bunch of tables.  I 
don't think there have been (m)any tests of that code on Solaris or with 
that many restore workers yet.


--
* Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


8.4 Performance improvements: was Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-13 Thread Jignesh K. Shah



Greg Smith wrote:

On Thu, 12 Mar 2009, Jignesh K. Shah wrote:

As soon as I get more "cycles" I will try variations of it but it 
would help if others can try it out in their own environments to see 
if it helps their instances.


What you should do next is see whether you can remove the bottleneck 
your test is running into via using a connection pooler.  That's what 
I think most informed people would do were you to ask how to setup an 
optimal environment using PostgreSQL that aimed to serve thousands of 
clients. If that makes your bottleneck go away, that's what you should 
be recommending to customers who want to scale in this fashion too.  
If the bottleneck moves to somewhere else, that new hot spot might be 
one people care more about.  Given that there are multiple good 
pooling solutions floating around already, it's hard to justify 
dumping coding and testing resources here if that makes the problem 
move somewhere else.


It's great that you've identified an alternate scheduling approach 
that helps on your problematic test case, but you're a long ways from 
having a full model of how changes to the locking model impact other 
database workloads.  As for the idea of doing something in this area 
for 8.4, there are a significant number of performance-related changes 
already committed for that version that deserve more focused testing 
during beta.  You're way too late to throw another one into that 
already crowded area.




On the other hand I have taken up a task of showing 8.4 Performance 
improvements over 8.3.
Can we do a vote on which specific performance features we want to test? 
I can use dbt2, dbt3 tests to see how 8.4 performs and compare it with 
8.3? Also if you have your own favorite test to test it out let me 
know.. I have allocated some time for this task so it is feasible for me 
to do this.


Many of the improvements may not be visible through this standard tests 
so feedback on testing methology for those is also appreciated.
* Visibility map - Reduce Vacuum overhead - (I think I can time vacuum 
with some usage on both databases)
* Prefetch IO with posix_fadvice () - Though I am not sure if it is 
supported on UNIX or not (but can be tested by standard tests)

* Parallel pg_restore (Can be tested with a big database dump)

Any more features that I can stress during the testing phase?



Regards,
Jignesh


--
* Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD


--
Jignesh Shah   http://blogs.sun.com/jkshah  
The New Sun Microsystems,Inc   http://sun.com/postgresql


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance