Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Tom Lane
Gavin Sherry <[EMAIL PROTECTED]> writes: > Could you demonstrate that point by showing us timings for shared_buffers > sizes from 512K up to, say, 2 MB? The two numbers you give there might > just have to do with managing a large buffer. Using PG CVS HEAD on 64-bit Intel Xeon (1MB L2 cache), Fedor

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Luke Lonergan
Gavin, Mark, > Could you demonstrate that point by showing us timings for > shared_buffers sizes from 512K up to, say, 2 MB? The two > numbers you give there might just have to do with managing a > large buffer. I suggest two experiments that we've already done: 1) increase shared buffers to d

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Gavin Sherry
On Mon, 5 Mar 2007, Mark Kirkwood wrote: > To add a little to this - forgetting the scan resistant point for the > moment... cranking down shared_buffers to be smaller than the L2 cache > seems to help *any* sequential scan immensely, even on quite modest HW: > > e.g: PIII 1.26Ghz 512Kb L2 cache,

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Mark Kirkwood
Tom Lane wrote: "Luke Lonergan" <[EMAIL PROTECTED]> writes: The issue is summarized like this: the buffer cache in PGSQL is not "scan resistant" as advertised. Sure it is. As near as I can tell, your real complaint is that the bufmgr doesn't attempt to limit its usage footprint to fit in L2 c

[HACKERS] Automatic adjustment of bgwriter_lru_maxpages (was: Dead Space Map version 2)

2007-03-04 Thread ITAGAKI Takahiro
"Jim C. Nasby" <[EMAIL PROTECTED]> wrote: > > > Perhaps it would be better to have the bgwriter take a look at how many > > > dead tuples (or how much space the dead tuples account for) when it > > > writes a page out and adjust the DSM at that time. > > > > Yeah, I feel it is worth optimizable,

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Luke Lonergan
One more thing: the L2 is invalidated when re-written from the kernel IO cache, but the pages addressed in L2 retain their values when 'writeen thru' which allows the new data to be re-used up the executor chain. - Luke Msg is shrt cuz m on ma treo -Original Message- From: Tom Lane

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Luke Lonergan
When we instrument the page selections made within the buffer cache, they are sequential and span the entire address space of the cache. With respect to whether it's L2, it's a conclusion based on the experimental results. It's not the TLB, as we also tested for the 512 entries for each L2. On

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Tom Lane
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > The issue is summarized like this: the buffer cache in PGSQL is not "scan > resistant" as advertised. Sure it is. As near as I can tell, your real complaint is that the bufmgr doesn't attempt to limit its usage footprint to fit in L2 cache; which is h

[HACKERS] Bug: Buffer cache is not scan resistant

2007-03-04 Thread Luke Lonergan
I'm putting this out there before we publish a fix so that we can discuss how best to fix it. Doug and Sherry recently found the source of an important performance issue with the Postgres shared buffer cache. The issue is summarized like this: the buffer cache in PGSQL is not "scan resistant" as

Re: [HACKERS] ERROR: operator does not exist: integer !=- integer

2007-03-04 Thread Joshua D. Drake
I missed the first post and can't seem to search for it - so correct me if I am missing something. Isn't the problem here a missing space? != is a valid operator and -1 is the value you are comparing to. !=-1 is not valid but != -1 is correct and what I assume you are looking to achieve.

Re: [HACKERS] New Access Method

2007-03-04 Thread Tom Lane
"Alan Gatt" <[EMAIL PROTECTED]> writes: > make[4]: Entering directory > `/c/dev/postgres/pgsql_tip/src/backend/access/mdim' > gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline > -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -g > -I../../../../src/include -I./src/includ

Re: [HACKERS] Synchronized Scan update

2007-03-04 Thread Joshua D. Drake
(2) sync_scan_offset: Start a new scan this many pages before a currently running scan to take advantage of the pages that are likely already in cache. I'm somewhat dubious about this parameter, I have to say, even though I am eager for this feature. It seems like a "magic" parameter tha

Re: [HACKERS] Trivial HugeTLB Benchmark

2007-03-04 Thread Tom Lane
Ryan Cumming <[EMAIL PROTECTED]> writes: > I ran each pgbench after a fresh reboot. I used 85 huge pages reserved at > boot for the huge page test, and none for the normal shared memory test. > Normal shared memory: > -bash-3.00$ pgbench -c 5 -t 1 > starting vacuum...end. > transaction type:

Re: [HACKERS] ERROR: operator does not exist: integer !=- integer

2007-03-04 Thread Tom Lane
"William ZHANG" <[EMAIL PROTECTED]> writes: > backend> select -1 !=-1; > ERROR: operator does not exist: integer !=- integer at character 11 This is not a bug. regards, tom lane ---(end of broadcast)--- TIP 6: explain analy

[HACKERS] Trivial HugeTLB Benchmark

2007-03-04 Thread Ryan Cumming
Hey, Out of curiosity I benchmarked PostgreSQL 8.2.3 using huge pages for shared memory. Oracle claims fairly significant speedups with huge pages but I couldn't find any information on PostgreSQL. I used the attached patch to enable huge pages on Linux. The test hardware is a dual Nocona Xeon

[HACKERS] ERROR: operator does not exist: integer !=- integer

2007-03-04 Thread William ZHANG
Here is the steps to reproduce it in CVS HEAD: $ uname -a Linux os-server 2.6.9-11.19AX #1 Fri Aug 5 05:12:07 EDT 2005 i686 i686 i386 GNU/Linux $ ./postgres --single -D $HOME/pgsql/data postgres PostgreSQL stand-alone backend 8.3devel backend> show server_version; 1: server_version

Re: [HACKERS] ERROR: operator does not exist: integer !=- integer

2007-03-04 Thread Shane Ambler
Andrew - Supernews wrote: On 2007-03-04, William ZHANG <[EMAIL PROTECTED]> wrote: Here is the steps to reproduce it in CVS HEAD: backend> select -1 !=-1; This arguably isn't a bug, because != is not a standard SQL operator, and therefore !=- can legitimately be defined as a single operator by

Re: [HACKERS] UPSERT

2007-03-04 Thread Martijn van Oosterhout
On Sun, Mar 04, 2007 at 02:55:47PM +0200, Hannu Krosing wrote: > Is'nt the standard way of doing it thus: > > UPDATE > IF NOT FOUND THEN > INSERT > IF DUPLICATE KEY THEN > UPDATE > END IF > END IF > > At least this is how UPSERT is usually done in plpgsql Well, you need to loop, becau

Re: [HACKERS] UPSERT

2007-03-04 Thread Petr Jelinek
Bruno Wolff III wrote: On Sun, Mar 04, 2007 at 14:55:47 +0200, Hannu Krosing <[EMAIL PROTECTED]> wrote: UPDATE IF NOT FOUND THEN INSERT IF DUPLICATE KEY THEN UPDATE END IF END IF I believe it is possible for the above to fail. For example another transaction could create a matching

Re: [HACKERS] UPSERT

2007-03-04 Thread Hannu Krosing
Ühel kenal päeval, P, 2007-03-04 kell 07:46, kirjutas Bruno Wolff III: > On Sun, Mar 04, 2007 at 14:55:47 +0200, > Hannu Krosing <[EMAIL PROTECTED]> wrote: > > > > UPDATE > > IF NOT FOUND THEN > > INSERT > > IF DUPLICATE KEY THEN > > UPDATE > > END IF > > END IF > > I believe it is p

Re: [HACKERS] UPSERT

2007-03-04 Thread Bruno Wolff III
On Sun, Mar 04, 2007 at 14:55:47 +0200, Hannu Krosing <[EMAIL PROTECTED]> wrote: > > UPDATE > IF NOT FOUND THEN > INSERT > IF DUPLICATE KEY THEN > UPDATE > END IF > END IF I believe it is possible for the above to fail. For example another transaction could create a matching record b

Re: [HACKERS] UPSERT

2007-03-04 Thread Hannu Krosing
Ühel kenal päeval, R, 2007-03-02 kell 10:13, kirjutas Tom Lane: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > My instinct would be to follow your first strategy, i.e. detect which > > path is needed rather than try one and then if it fails do the other. > > The very first thing you need to thi

Re: [HACKERS] Synchronized Scan update

2007-03-04 Thread Simon Riggs
On Fri, 2007-03-02 at 15:03 -0800, Jeff Davis wrote: > Is there any consensus about whether to include these two parameters as > GUCs or constants if my patch is to be accepted? > > (1) sync_scan_threshold: Use synchronized scanning for tables greater > than this many pages; smaller tables will no

Re: [HACKERS] ERROR: operator does not exist: integer !=- integer

2007-03-04 Thread Andrew - Supernews
On 2007-03-04, William ZHANG <[EMAIL PROTECTED]> wrote: > Here is the steps to reproduce it in CVS HEAD: > backend> select -1 !=-1; This arguably isn't a bug, because != is not a standard SQL operator, and therefore !=- can legitimately be defined as a single operator by the user. -- Andrew, Sup

Re: [HACKERS] New Access Method

2007-03-04 Thread Martijn van Oosterhout
On Sun, Mar 04, 2007 at 09:32:58AM +0100, Alan Gatt wrote: > Also, any pointers where I can find the operator classes and functions for > the GIST index? The documentation is a good start: http://www.postgresql.org/docs/8.1/static/gist.html There are also plenty of examples in the contrib portio

Re: [HACKERS] New Access Method

2007-03-04 Thread Alan Gatt
Also, any pointers where I can find the operator classes and functions for the GIST index? Thanks On 04/03/07, Alan Gatt <[EMAIL PROTECTED]> wrote: Ok, so I am using GCC 3.2 as a compiler, and the following is the error message: make[4]: Entering directory `/c/dev/postgres/pgsql_tip/src/backe