Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread Bruno Wolff III
On Sat, Feb 19, 2005 at 18:04:42 -0500, Now, lets imagine PostgreSQL is being developed by a large company. QA announces it has found a bug that will cause all the users data to disappear if they don't run a maintenence program correctly. Vacuuming one or two tables is not enough, you have

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread pgsql
On Sat, Feb 19, 2005 at 18:04:42 -0500, Now, lets imagine PostgreSQL is being developed by a large company. QA announces it has found a bug that will cause all the users data to disappear if they don't run a maintenence program correctly. Vacuuming one or two tables is not enough, you have

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread Robert Treat
On Sunday 20 February 2005 00:30, Tom Lane wrote: Mark Kirkwood [EMAIL PROTECTED] writes: To be fair to Mark, there does seem to be an increasing number of reports of this issue. In spite of the in-the-works fix for 8.1, it would be a pity to see customers losing data from xid wrap-around.

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread Stephan Szabo
On Sun, 20 Feb 2005 [EMAIL PROTECTED] wrote: On Sat, Feb 19, 2005 at 18:04:42 -0500, Now, lets imagine PostgreSQL is being developed by a large company. QA announces it has found a bug that will cause all the users data to disappear if they don't run a maintenence program correctly.

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread pgsql
On Sun, 20 Feb 2005 [EMAIL PROTECTED] wrote: On Sat, Feb 19, 2005 at 18:04:42 -0500, Now, lets imagine PostgreSQL is being developed by a large company. QA announces it has found a bug that will cause all the users data to disappear if they don't run a maintenence program correctly.

Re: [HACKERS] Get rid of system attributes in pg_attribute?

2005-02-20 Thread Robert Treat
On Sunday 20 February 2005 00:25, Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: If I am understanding this correctly, they could only be displayed if selected explicitly right? That's always been true. The behavior at the level of SQL commands wouldn't change. The question is

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread Stephan Szabo
On Sun, 20 Feb 2005 [EMAIL PROTECTED] wrote: On Sun, 20 Feb 2005 [EMAIL PROTECTED] wrote: On Sat, Feb 19, 2005 at 18:04:42 -0500, Now, lets imagine PostgreSQL is being developed by a large company. QA announces it has found a bug that will cause all the users data to

Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-20 Thread Magnus Hagander
Magnus prepared a trivial patch which added the O_SYNC flag for windows and mapped it to FILE_FLAG_WRITE_THROUGH in win32_open.c. Attached is this trivial patch. As Merlin says, it needs some more reliability testing. But the numbers are at least reasonable - it *seems* like it's doing the

Re: [HACKERS] Get rid of system attributes in pg_attribute?

2005-02-20 Thread Tom Lane
Robert Treat [EMAIL PROTECTED] writes: One of us is not understanding the other :-) I'm asking if I have a piece of code that does something like select attname from pg_attribute where attrelid = 'stock'::regclass::oid with the intent of displaying all those attnames, then the system atts

Re: [HACKERS] Fwd: Apple Darwin disabled fsync?

2005-02-20 Thread Greg Stark
Peter Bierman [EMAIL PROTECTED] writes: In most cases you do not need such a heavy handed operation and fsync() is good enough. Really? Can you think of a single application for which this definition of fsync is useful? Kernel buffers are transparent to the application, just as the disk

[HACKERS] SMP buffer management test question

2005-02-20 Thread lsunley
There was a mention of a context-swap-storm test in the posts on the ARC patent. Where might I obtain a copy of this test procedure? Thanks -- --- [EMAIL PROTECTED] ---

Re: [HACKERS] win32 performance - fsync question

2005-02-20 Thread Magnus Hagander
Portability, or rather the complete lack of it. Stuff that isn't in the Single Unix Spec is a hard sell. O_DIRECT is reasonably common among modern Unixen (it is supported by Linux, FreeBSD, and probably a couple of the commercial variants like AIX or IRIX); it should also be reasonably

Re: [HACKERS] SMP buffer management test question

2005-02-20 Thread Tom Lane
[EMAIL PROTECTED] writes: There was a mention of a context-swap-storm test in the posts on the ARC patent. Where might I obtain a copy of this test procedure? http://archives.postgresql.org/pgsql-performance/2004-04/msg00280.php (The archiver seems to have lost the message's separation into

Re: [HACKERS] Query optimizer 8.0.1 (and 8.0)

2005-02-20 Thread Bruce Momjian
Jim C. Nasby wrote: On Mon, Feb 14, 2005 at 09:55:38AM -0800, Ron Mayer wrote: I still suspect that the correct way to do it would not be to use the single correlation, but 2 stats - one for estimating how sequential/random accesses would be; and one for estimating the number of pages

Re: [HACKERS] win32 performance - fsync question

2005-02-20 Thread Magnus Hagander
One point that I no longer recall the reasoning behind is that xlog.c doesn't think O_SYNC is a preferable default over fsync. For larger (8k) transactions O_SYNC|O_DIRECT is only good with the recent pending patch to group WAL writes together. The fsync method gives the OS a chance to do

Re: [HACKERS] Data loss, vacuum, transaction wrap-around

2005-02-20 Thread Mark Kirkwood
Tom Lane wrote: The question is whether we are willing to back-patch a fairly large amount of not-very-well-tested code into 8.0. See http://archives.postgresql.org/pgsql-patches/2005-02/msg00123.php http://archives.postgresql.org/pgsql-committers/2005-02/msg00127.php

Re: [HACKERS] Query optimizer 8.0.1 (and 8.0)

2005-02-20 Thread Mark Kirkwood
Bruce Momjian wrote: Jim C. Nasby wrote: On Mon, Feb 14, 2005 at 09:55:38AM -0800, Ron Mayer wrote: I still suspect that the correct way to do it would not be to use the single correlation, but 2 stats - one for estimating how sequential/random accesses would be; and one for estimating the number

[HACKERS] Can we remove SnapshotSelf?

2005-02-20 Thread Tom Lane
As of CVS tip, there is no code in the system that uses SnapshotSelf. I am wondering if we can get rid of it and thereby save one test in the heavily used HeapTupleSatisfiesVisibility() macro. There is one place in the foreign-key triggers that uses the underlying HeapTupleSatisfiesItself() test

Re: [HACKERS] SMP buffer management test question

2005-02-20 Thread lsunley
Thanks Well I tried it out and on OS/2 using 8.0 it has the same problems as on Linux, with two clients running on a dual PIII 750 both CPUs go to 100% and the system is madly switching between the two processes. With a single client, about 35% CPU is used. Should the new buffer manager patch

Re: [HACKERS] SMP buffer management test question

2005-02-20 Thread Tom Lane
[EMAIL PROTECTED] writes: Well I tried it out and on OS/2 using 8.0 it has the same problems as on Linux, with two clients running on a dual PIII 750 both CPUs go to 100% and the system is madly switching between the two processes. With a single client, about 35% CPU is used. Should the new

Re: [HACKERS] Fwd: Apple Darwin disabled fsync?

2005-02-20 Thread Peter Bierman
At 12:38 AM -0500 2/20/05, Tom Lane wrote: Dominic Giampaolo [EMAIL PROTECTED] writes: I believe that what the above comment refers to is the fact that fsync() is not sufficient to guarantee that your data is on stable storage and on MacOS X we provide a fcntl(), called F_FULLFSYNC, to ask the

[HACKERS] Time Zone Names Problem

2005-02-20 Thread Curt Sampson
I'm running PostgreSQL 8.0.1 from the NetBSD pkgsrc package. I'm getting some odd behavior with time zone names: SET timezone TO 'JST'; psql:z.sql:1: ERROR: unrecognized time zone name: JST SET timezone TO 'EST'; psql:z.sql:2: ERROR: unrecognized time zone name: EST SET

Re: [HACKERS] postgres crashing on a seemingly good query

2005-02-20 Thread Neil Conway
On Sat, 2005-02-19 at 14:35 -0500, Tom Lane wrote: Good catch. I've applied the attached patch (this is against 8.0/CVS tip but applies with some fuzz to 7.4). Is there a way to repro this via SQL? (It would be nice to have a regression test...) -Neil ---(end of

Re: [HACKERS] postgres crashing on a seemingly good query

2005-02-20 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: On Sat, 2005-02-19 at 14:35 -0500, Tom Lane wrote: Good catch. I've applied the attached patch (this is against 8.0/CVS tip but applies with some fuzz to 7.4). Is there a way to repro this via SQL? (It would be nice to have a regression test...) No,

Re: [HACKERS] Time Zone Names Problem

2005-02-20 Thread Tom Lane
Curt Sampson [EMAIL PROTECTED] writes: I'm running PostgreSQL 8.0.1 from the NetBSD pkgsrc package. I'm getting some odd behavior with time zone names: SET timezone TO 'EST'; psql:z.sql:2: ERROR: unrecognized time zone name: EST SET timezone TO 'Asia/Tokyo';

Re: [HACKERS] Fwd: Apple Darwin disabled fsync?

2005-02-20 Thread Greg Stark
Peter Bierman [EMAIL PROTECTED] writes: I think the intent of fsync() is closer to what you describe, but the convention is that fsync() hands responsibility to the disk hardware. The convention was also that the hardware didn't confirm the command until it had actually been executed... None

Re: [HACKERS] Time Zone Names Problem

2005-02-20 Thread Curt Sampson
On Sun, 20 Feb 2005, Tom Lane wrote: Hmm, all of those should work (and do work here). Are the corresponding files present in the appropriate installation directory? Look under .../share/postgresql/timezone/ They are not; it's evidently a problem with the way the NetBSD package installs things.

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-20 Thread Bruce Momjian
Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The advantage of using a counter instead of a simple active bit is that buffers that are (or have been) used heavily will be able to go through several sweeps of the clock before being freed. Infrequently used buffers (such as those

Re: [HACKERS] Query optimizer 8.0.1 (and 8.0)

2005-02-20 Thread pgsql
Jim C. Nasby wrote: On Mon, Feb 14, 2005 at 09:55:38AM -0800, Ron Mayer wrote: I still suspect that the correct way to do it would not be to use the single correlation, but 2 stats - one for estimating how sequential/random accesses would be; and one for estimating the number of pages

[HACKERS] Transaction Aborted

2005-02-20 Thread Edwin S. Ramirez
Hello, This may sound familiar since there was a discussion before... I created an importing tool which allows users to load data into their tables using a graphical interface. The import operations occur within a transaction so that the user gets a chance to change their mind. After the

Re: [HACKERS] Strange RETURN NEXT behaviour in Postgres 8.0

2005-02-20 Thread Sergey E. Koposov
Sergey E. Koposov [EMAIL PROTECTED] writes: LOOP FETCH cur into rec; RETURN NEXT rec; EXIT WHEN NOT FOUND; END LOOP; RETURN; Don't you think you should have the EXIT *above* the RETURN NEXT? I would expect this to emit a bogus row of nulls after the last

Re: [HACKERS] Transaction Aborted

2005-02-20 Thread Jeff Davis
I think that the feature you're looking for is a SAVEPOINT. After the import, create a savepoint, and then if the user makes a typo revert to that savepoint. Regards, Jeff Davis On Sun, 2005-02-20 at 22:05 -0500, Edwin S. Ramirez wrote: Hello, This may sound familiar since there