Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-29 Thread Zeugswetter Andreas DCP SD
> DB2: > Uncommitted Read (UR) mode "Dirty read" isn't the default, or > the recommended lock level for most apps. I was considering > Cursor Stability mode (or higher), which is the default Sorry, they call it "read committed" but actually do cursor stability, which does keep one lock on the

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-29 Thread simon
>> > OTOH DB2 and SQLServer take block level >> > read locks, so they can do this too, but at major loss of >concurrency >> > and threat of deadlock. > >Note, that in the usual committed read isolation, they do not need to >read lock a row ! e.g. Informix only verifies, that it could lock the >ro

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-29 Thread Zeugswetter Andreas DCP SD
> > OTOH DB2 and SQLServer take block level > > read locks, so they can do this too, but at major loss of concurrency > > and threat of deadlock. Note, that in the usual committed read isolation, they do not need to read lock a row ! e.g. Informix only verifies, that it could lock the row (tha

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-28 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > (What would be the profiler to use on FBSD?) gprof should work fine. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-28 Thread Jim C. Nasby
On Fri, Nov 25, 2005 at 10:20:11AM -0500, Tom Lane wrote: > Qingqing Zhou <[EMAIL PROTECTED]> writes: > > I can see your computer is really slow, so my theory is that since it is > > easy to hold a running-slowly horse than a fast one, so my spinlock on a > > 2.4G modern machine should takes relati

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-27 Thread Christopher Kings-Lynne
Thinking more about other systems, ISTM that Oracle can do this, as can any MVCC based system. OTOH DB2 and SQLServer take block level read locks, so they can do this too, but at major loss of concurrency and threat of deadlock. Having said that, *any* system that chose not to do this would be sev

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: >>> ...and for emphasis: this optimization of SeqScans is not possible with >>> any other database system, so its a big win for PostgreSQL. > Why is it spin to call it a big win? I didn't say it wasn't a big win; it was the first part of the sentence that b

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Luke Lonergan
Title: Re: [HACKERS] gprof SELECT COUNT(*) results Nice job Qingqing and Tom! The improved executor / agg performance will likely help BI / data warehouse customers a lot.  I’ll get some DBT-3 results to substantiate as soon as we can. - Luke On 11/26/05 12:13 AM, "Qingqing Zhou&quo

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Simon Riggs
On Sat, 2005-11-26 at 11:53 -0500, Tom Lane wrote: > Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: > >> ...and for emphasis: this optimization of SeqScans is not possible with > >> any other database system, so its a big win for PostgreSQL. > > > With any other db system? That's a big call.

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Tom Lane
Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: >> ...and for emphasis: this optimization of SeqScans is not possible with >> any other database system, so its a big win for PostgreSQL. > With any other db system? That's a big call. Why? One could equally well spin it negatively, as "this o

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Christopher Kings-Lynne
...and for emphasis: this optimization of SeqScans is not possible with any other database system, so its a big win for PostgreSQL. With any other db system? That's a big call. Why? Not even other MVCC systems? Chris ---(end of broadcast)---

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Simon Riggs
On Sat, 2005-11-26 at 03:13 -0500, Qingqing Zhou wrote: > "Tom Lane" <[EMAIL PROTECTED]> wrote > > > > I plan to take a look at it soon. > > > > "Stand corrected"[Merlin]! In-memory "SELECT COUNT(*)" doubles the > performance due to my test. > > - before - > 1.56 s > > - now - > 0.72 s > ...a

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-26 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> wrote > > I plan to take a look at it soon. > "Stand corrected"[Merlin]! In-memory "SELECT COUNT(*)" doubles the performance due to my test. - before - 1.56 s - now - 0.72 s Regards, Qingqing ---(end of broadcast)-

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-25 Thread Qingqing Zhou
On Fri, 25 Nov 2005, Tom Lane wrote: > > Is that "modern machine" a Xeon by any chance? > $#cat /proc/cpuinfo | grep "model name" model name : Intel(R) Pentium(R) 4 CPU 2.40GHz I can find a 4way Xeon (but it is shared by many users): /h/164/zhouqq#cat /proc/cpuinfo |grep "model name" model

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-25 Thread Olivier Thauvin
Le Vendredi 25 Novembre 2005 16:20, Tom Lane a écrit : > Qingqing Zhou <[EMAIL PROTECTED]> writes: > > I can see your computer is really slow, so my theory is that since it is > > easy to hold a running-slowly horse than a fast one, so my spinlock on a > > 2.4G modern machine should takes relativel

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-25 Thread Simon Riggs
On Fri, 2005-11-25 at 09:54 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2005-11-24 at 23:48 -0500, Tom Lane wrote: > >> Yeah, I was wondering the same. It'd be possible to rewrite the seqscan > >> stuff so that we do the visibility tests for all the tuples on a giv

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-25 Thread Tom Lane
Qingqing Zhou <[EMAIL PROTECTED]> writes: > I can see your computer is really slow, so my theory is that since it is > easy to hold a running-slowly horse than a fast one, so my spinlock on a > 2.4G modern machine should takes relatively longer time to get effective. > Just kidding. Is that "moder

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Thu, 2005-11-24 at 23:48 -0500, Tom Lane wrote: >> Yeah, I was wondering the same. It'd be possible to rewrite the seqscan >> stuff so that we do the visibility tests for all the tuples on a given >> page at once, taking the buffer content lock just onc

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-25 Thread Simon Riggs
On Thu, 2005-11-24 at 23:48 -0500, Tom Lane wrote: > > What's more, we can see that for each row, a LWLock pair is invoked. So on > > a more aggressive thought, can we change it to page level? > > Yeah, I was wondering the same. It'd be possible to rewrite the seqscan > stuff so that we do the v

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Qingqing Zhou
On Thu, 24 Nov 2005, Tom Lane wrote: > > I don't see those costing nearly as much as your results show > ... perhaps there's something platform-specific at work? > What I see, down to the 1% level, is > I can see your computer is really slow, so my theory is that since it is easy to hold a runni

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Qingqing Zhou
On Thu, 24 Nov 2005, Qingqing Zhou wrote: > > I may need to write some separate tests to see if this is what we should > pay for bus lock instruction. > Here I come up with a test program to see how spinlock costs: $/pgsql/src/backend/storage/lmgr#./a.out Spinlock pair(2648542) duration: 143.13

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Tom Lane
Qingqing Zhou <[EMAIL PROTECTED]> writes: > Yeah, understood. What I can't understand that in this case why it costs > so much -- without concurrency, the LWLock code path just invloves > spinlock_lock/unlock and serveral simple instructions? I don't see those costing nearly as much as your result

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Qingqing Zhou
On Thu, 24 Nov 2005, Greg Stark wrote: > > > You executed LWLock 2.6 million times in just under 300ms. If my math is right > that's about 115 nanoseconds per lock or about 300 cycles on a 2.6Ghz > processor. > > That sounds like a lot but it's about the right order of magnitude. Was this > on a

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Greg Stark
Qingqing Zhou <[EMAIL PROTECTED]> writes: > Yeah, understood. What I can't understand that in this case why it costs > so much -- without concurrency, the LWLock code path just invloves > spinlock_lock/unlock and serveral simple instructions? You executed LWLock 2.6 million times in just under 3

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Qingqing Zhou
On Thu, 24 Nov 2005, Simon Riggs wrote: > > Maybe, maybe not. The whole system is designed around high levels of > concurrent access. If you know for certain you don't ever need that then > other systems are probably the right choice. Concurrency has a cost and > a benefit. If you measure the co

Re: [HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Simon Riggs
On Thu, 2005-11-24 at 13:25 -0500, Qingqing Zhou wrote: > I did some gprof on a simple "SELECT COUNT(*) FROM test" query on cvs tip. > > Linux josh.db 2.4.29-1 #2 Tue Jan 25 17:03:33 EST 2005 i686 unknown > gcc: 2.96 > gprof: 2.13.90.0.2 > ./configure --without-readline > > There are 260k or so r

[HACKERS] gprof SELECT COUNT(*) results

2005-11-24 Thread Qingqing Zhou
I did some gprof on a simple "SELECT COUNT(*) FROM test" query on cvs tip. Linux josh.db 2.4.29-1 #2 Tue Jan 25 17:03:33 EST 2005 i686 unknown gcc: 2.96 gprof: 2.13.90.0.2 ./configure --without-readline There are 260k or so records in table test(i int), about 1500 pages. I give a shared_buffers