Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Mark Wong <[EMAIL PROTECTED]> writes: > After over a year of problems (old site > http://developer.osdl.org/markw/postgrescvs/) I have resumed producing > daily results of dbt-2 against PostgreSQL CVS code with results here: > http://dbt.osdl.org/dbt2.html This is good to hear! I am curious where we are now compared to where we were a year ago ... do you still have the old data, and is the test setup still comparable? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
> Mark Wong <[EMAIL PROTECTED]> writes: >> After over a year of problems (old site >> http://developer.osdl.org/markw/postgrescvs/) I have resumed producing >> daily results of dbt-2 against PostgreSQL CVS code with results here: >> http://dbt.osdl.org/dbt2.html > > This is good to hear! I am curious where we are now compared to where > we were a year ago ... do you still have the old data, and is the test > setup still comparable? The test setup is on completely different hardware. I still have the old data and it's accessible, but it'll take a little bit of work to regenerate the links. I'll try to work on that. Mark ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
[EMAIL PROTECTED] wrote: Mark Wong <[EMAIL PROTECTED]> writes: After over a year of problems (old site http://developer.osdl.org/markw/postgrescvs/) I have resumed producing daily results of dbt-2 against PostgreSQL CVS code with results here: http://dbt.osdl.org/dbt2.html This is good to hear! I am curious where we are now compared to where we were a year ago ... do you still have the old data, and is the test setup still comparable? The test setup is on completely different hardware. I still have the old data and it's accessible, but it'll take a little bit of work to regenerate the links. I'll try to work on that. I think it would also help if you would create reference runs for the latest 8.0 and 8.1 releases on the new hardware. Best Regards Michael Paesold ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: >> After over a year of problems (old site >> http://developer.osdl.org/markw/postgrescvs/) I have resumed producing >> daily results of dbt-2 against PostgreSQL CVS code with results here: >> http://dbt.osdl.org/dbt2.html > > This is good to hear! I am curious where we are now compared to where > we were a year ago ... do you still have the old data, and is the test > setup still comparable? I made another couple of gross mistakes of forgetting to compile PostgreSQL with --enable-thread-safe and enabling the user space irq balancing program in Linux. I've restarted the histories with 600 and 800 warehouse runs where 600 should be below peak system performance and 800 pushing the system beyond it's peak performance. Still working on getting pointers to the old data... Regards, Mark ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Michael Paesold wrote: > [EMAIL PROTECTED] wrote: >>> Mark Wong <[EMAIL PROTECTED]> writes: After over a year of problems (old site http://developer.osdl.org/markw/postgrescvs/) I have resumed producing daily results of dbt-2 against PostgreSQL CVS code with results here: http://dbt.osdl.org/dbt2.html >>> This is good to hear! I am curious where we are now compared to where >>> we were a year ago ... do you still have the old data, and is the test >>> setup still comparable? >> >> The test setup is on completely different hardware. I still have the old >> data and it's accessible, but it'll take a little bit of work to >> regenerate the links. I'll try to work on that. > > I think it would also help if you would create reference runs for the > latest 8.0 and 8.1 releases on the new hardware. Ok, I'll try to work those in within the next couple days. Regards, Mark ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
On Sun, Oct 08, 2006 at 05:26:11PM -0700, Mark Wong wrote: > I made another couple of gross mistakes of forgetting to compile > PostgreSQL with --enable-thread-safe and enabling the user space irq > balancing program in Linux. I've restarted the histories with 600 and What's the advantage of irq balancing in user space as opposed to the kernel (which is the default, right?) -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Jim C. Nasby wrote: On Sun, Oct 08, 2006 at 05:26:11PM -0700, Mark Wong wrote: I made another couple of gross mistakes of forgetting to compile PostgreSQL with --enable-thread-safe and enabling the user space irq balancing program in Linux. I've restarted the histories with 600 and What's the advantage of irq balancing in user space as opposed to the kernel (which is the default, right?) Linux doesn't have irq balancing in the kernel. They've made the decision to leave that to a user space process. The irq balancing flag in the kernel is just to enable the hooks for a user space program. Mark ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
On Mon, Oct 09, 2006 at 10:37:32AM -0700, Mark Wong wrote: > Jim C. Nasby wrote: > >On Sun, Oct 08, 2006 at 05:26:11PM -0700, Mark Wong wrote: > >>I made another couple of gross mistakes of forgetting to compile > >>PostgreSQL with --enable-thread-safe and enabling the user space irq > >>balancing program in Linux. I've restarted the histories with 600 and > > > >What's the advantage of irq balancing in user space as opposed to the > >kernel (which is the default, right?) > > Linux doesn't have irq balancing in the kernel. They've made the > decision to leave that to a user space process. The irq balancing flag > in the kernel is just to enable the hooks for a user space program. Oooh, interesting... I wonder how many installs out there are running without IRQ balancing enabled. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Luke Lonergan wrote: +1 Mark, can you quantify the impact of not running with IRQ balancing enabled? Yeah, I'll try to have that done within a couple of days. Mark ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Luke Lonergan wrote: +1 Mark, can you quantify the impact of not running with IRQ balancing enabled? Whoops, look like performance was due more to enabling the --enable-thread-safe flag. IRQ balancing on : 7086.75 http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/158/ IRQ balancing off: 7057.90 http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/163/ The interrupt charts look completely different. There's too much stuff on the chart to determine what interrupts are from what though. :( It needs to be redone per processor (as opposed to per interrupt per processor) to be more useful in determining if one processor is overloaded due to interrupts. http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/158/report/sar/sar-intr.png http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/163/report/sar/sar-intr.png But the sum of all the interrupts handled are close between tests so it seems clear no single processor was overloaded: http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/158/report/sar/sar-intr_s.png http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/163/report/sar/sar-intr_s.png Mark ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] continuing daily testing of dbt2 against postgresql
Yeah, I'm sure binding each process to a CPU would be a significant help. Something I've always wanted to quantify but haven't made time for... Mark Luke Lonergan wrote: One of our customers noticed that there were a high number of NUMA cache misses on a quad core opteron system running Bizgres MPP resulting in about a 15% performance hit. We use a process-based parallelization approach and we can guess that there's context switching due to the high degree of pipeline parallelism in our executions plans. Each context switch likely switches a process away from the CPU with local memory, resulting in the NUMA cache misses. The answer for us is to bind each process to a CPU. Might that help in running DBT-2? - Luke On 10/10/06 9:40 AM, "Mark Wong" <[EMAIL PROTECTED]> wrote: Luke Lonergan wrote: +1 Mark, can you quantify the impact of not running with IRQ balancing enabled? Whoops, look like performance was due more to enabling the --enable-thread-safe flag. IRQ balancing on : 7086.75 http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/158/ IRQ balancing off: 7057.90 http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/163/ The interrupt charts look completely different. There's too much stuff on the chart to determine what interrupts are from what though. :( It needs to be redone per processor (as opposed to per interrupt per processor) to be more useful in determining if one processor is overloaded due to interrupts. http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/158/report/sar/sar-intr.png http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/163/report/sar/sar-intr.png But the sum of all the interrupts handled are close between tests so it seems clear no single processor was overloaded: http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/158/report/sar/sar-intr_s.png http://dbt.osdl.org/dbt/dbt2dev/results/dev4-015/163/report/sar/sar-intr_s.png Mark ---(end of broadcast)--- TIP 6: explain analyze is your friend
