Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Thu, 2004-07-15 at 02:43, Mark Kirkwood wrote: I noticed that compiling with 5_1 patch applied fails due to XLOG_archive_dir being removed from xlog.c , but src/backend/commands/tablecmds.c still uses it. I did the following to tablecmds.c : 5408c5408 extern char

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Thu, 2004-07-15 at 03:02, Bruce Momjian wrote: I talked to Tom on the phone today and and I think we have a procedure for doing backup/restore in a fairly foolproof way. As outlined below, we need to record the start/stop and checkpoint WAL file names and offsets, and somehow pass those

Re: [HACKERS] [PATCHES] serverlog rotation/functions

2004-07-15 Thread Andreas Pflug
Bruce Momjian wrote: OK, I talked to Tom about this patch and I understand the issues now. I think the best solution will be to have the postmaster start a child process that can read the guc variables and create a log file based on it contents. The child would be responsible to create a new log

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Mark Kirkwood
I tried what I thought was a straightforward scenario, and seem to have broken it :-( Here is the little tale 1) initdb 2) set archive_mode and archive_dest in postgresql.conf 3) startup 4) create database called 'test' 5) connect to 'test' and type 'checkpoint' 6) backup PGDATA using 'tar

Re: [HACKERS] Release planning

2004-07-15 Thread Gaetano Mendola
Christopher Browne wrote: A long time ago, in a galaxy far, far away, Gaetano Mendola [EMAIL PROTECTED] wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Zeugswetter Andreas SB SD
Other db's have commands for: start/end external backup I see that the analogy to external backup was not good, since you are correct that dba's would expect that to stop all writes, so they can safely split their mirror or some such. Usually the expected time from start until end external

[HACKERS] How to display privileges in psql

2004-07-15 Thread Peter Eisentraut
Maybe other people want to comment on this issue. So far, \z and \dp have shown privileges of tables. Now \dn+ and \db+ show the privileges of schemas and tablespaces. Should we, for consistency, move the table privilege display to \dt+, or should we move the schema and tablespace display to,

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread HISADAMasaki
Dear Simon, I've just tested pitr_v5_2.patch and got an error message during archiving process as follows. -- begin LOG: archive command=cp /usr/local/pgsql/data/pg_xlog/ /tmp,return code=-1 -- end The command called in system(3) works, but it returns -1. system(3) can not

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Zeugswetter Andreas SB SD
Sorry for the stupid question, but how do I get this patch if I do not receive the patches mails ? The web interface html'ifies it, thus making it unusable. Thanks Andreas ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ?

Re: [HACKERS] [PATCHES] serverlog rotation/functions

2004-07-15 Thread Bruce Momjian
Andreas Pflug wrote: Bruce Momjian wrote: OK, I talked to Tom about this patch and I understand the issues now. I think the best solution will be to have the postmaster start a child process that can read the guc variables and create a log file based on it contents. The child would be

Re: [HACKERS] How to display privileges in psql

2004-07-15 Thread Bruce Momjian
Peter Eisentraut wrote: Maybe other people want to comment on this issue. So far, \z and \dp have shown privileges of tables. Now \dn+ and \db+ show the privileges of schemas and tablespaces. Should we, for consistency, move the table privilege display to \dt+, or should we move the

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Bruce Momjian
Simon Riggs wrote: On Thu, 2004-07-15 at 03:02, Bruce Momjian wrote: I talked to Tom on the phone today and and I think we have a procedure for doing backup/restore in a fairly foolproof way. As outlined below, we need to record the start/stop and checkpoint WAL file names and offsets,

Re: [HACKERS] Release planning

2004-07-15 Thread Marc G. Fournier
On Thu, 15 Jul 2004, Gaetano Mendola wrote: I'm sorry to see Postgresql releases driven by advertisment instead by good sense ( as it was till today ). The releases are not being driving by advertisement ... note that the decisions for including the features you list above was made before the

Re: [HACKERS] How to display privileges in psql

2004-07-15 Thread Alvaro Herrera
On Thu, Jul 15, 2004 at 10:41:49AM -0400, Bruce Momjian wrote: Peter Eisentraut wrote: Maybe other people want to comment on this issue. So far, \z and \dp have shown privileges of tables. Now \dn+ and \db+ show the privileges of schemas and tablespaces. Should we, for

Re: [HACKERS] How to display privileges in psql

2004-07-15 Thread Bruce Momjian
Alvaro Herrera wrote: On Thu, Jul 15, 2004 at 10:41:49AM -0400, Bruce Momjian wrote: Peter Eisentraut wrote: Maybe other people want to comment on this issue. So far, \z and \dp have shown privileges of tables. Now \dn+ and \db+ show the privileges of schemas and tablespaces.

[HACKERS] SysLogger subprocess

2004-07-15 Thread Andreas Pflug
Here's the core of the logger subprocess. - no rotation code so far, all syslogFile handling is for testing only - send_message_to_serverlog needs some careful handling of stderr, in case pipe writing fails or if after a log process restart redirecting stderr fails. In these cases, the original

Re: [HACKERS] [PATCHES] serverlog rotation/functions

2004-07-15 Thread Andreas Pflug
Bruce Momjian wrote: I don't see any reason to have a pattern though I suppose if you mix pgsql log files in with other log files it might be a problem. One idea would be for the client-side program to do some processing like this: SELECT * FROM dir_listing('/var/log') AS dir

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Bruce Momjian
Simon Riggs wrote: On Wed, 2004-07-14 at 10:57, Zeugswetter Andreas SB SD wrote: The recovery mechanism doesn't rely upon you knowing 1 or 3. The recovery reads pg_control (from the backup) and then attempts to de-archive the appropriate xlog segment file and then starts rollforward

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Thu, 2004-07-15 at 10:47, Mark Kirkwood wrote: I tried what I thought was a straightforward scenario, and seem to have broken it :-( Here is the little tale 1) initdb 2) set archive_mode and archive_dest in postgresql.conf 3) startup 4) create database called 'test' 5) connect to

Re: [HACKERS] How to display privileges in psql

2004-07-15 Thread Dennis Bjorklund
On Thu, 15 Jul 2004, Alvaro Herrera wrote: IMHO the \dn+ output would get too wide if you do that. I'd be in favor of using \zletter to display permissions of the object in \dletter I think it's time to start with longer command names. Tab completion makes it easy anyway. The short ones

[HACKERS] Nested Savepoints in Db2 Stinger

2004-07-15 Thread Simon Riggs
FYI http://www.databasejournal.com/features/db2/article.php/3361941 Best Regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so

Re: [HACKERS] SysLogger subprocess

2004-07-15 Thread Alvaro Herrera
On Thu, Jul 15, 2004 at 09:00:50PM +0200, Andreas Pflug wrote: While looking at pgstat.c to see how to peek for pipe data, I found readpipe=pgStatPipe[0]; select(readPipe+1, ..); which is probably usually the same as select(pgStatPipe[1], ..) This fd arithmetics seem a bit

Re: [HACKERS] possibly updating techdocs; mysql2pgsql on gborg

2004-07-15 Thread Robert Treat
On Tuesday 13 July 2004 09:06, Bruce Momjian wrote: Josh Berkus wrote: Robert, Bruce, If anybody still has access to that page, the project has moved to gborg specifically over to http://gborg.postgresql.org/project/mysql2psql/projdisplay.php where a new version

Re: [HACKERS] SysLogger subprocess

2004-07-15 Thread Andreas Pflug
Alvaro Herrera wrote: No, it's not pgStatPipe[1], it's select(2)'s first argument; max fd in the sets plus one. Probably your code works because pgStatPipe[1] == pgStatPipe[0] + 1. Ah, I see now, thanks. Regards, Andreas ---(end of broadcast)--- TIP

Re: [HACKERS] possibly updating techdocs; mysql2pgsql on gborg

2004-07-15 Thread Josh Berkus
Robert, Maybe this should be removed from contrib? We agreed to remove the mysql folder from contrib about 6 months ago, on Hackers. -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 9: the planner will

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Magnus Hagander
It occurs to me that with a check this thorough, we might be able to finesse the problem on Windows with the system returning very nonstandard timezone abbreviations. That is, we might simply #ifndef WIN32 the matching of zone names in try_timezone(). However I do not know whether this would

Re: [HACKERS] Is trust really a good default?

2004-07-15 Thread Peter Eisentraut
Magnus Hagander wrote: Certainly, I'm not saying it shuold change (I've given that up by now). But the difference would be that if you used -W with initdb, it would change the default *for that installation*. Initdb-with-no-parameters would stay the same to keep people who don't know about

[HACKERS] Note about robustness of transaction-related data structures

2004-07-15 Thread Tom Lane
While running a parallel regression test I saw a failure relation deleted while still in use out of the subtransactions test, after which the backend dumped core. The coredump was in AtSubAbort_smgr, and it was failing because upperPendingDeletes was NIL (which it should not have been

Re: [HACKERS] Is trust really a good default?

2004-07-15 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: Magnus Hagander wrote: Certainly, I'm not saying it shuold change (I've given that up by now). But the difference would be that if you used -W with initdb, it would change the default *for that installation*. The fallacy with this line of thought is

Re: [HACKERS] Is trust really a good default?

2004-07-15 Thread Peter Eisentraut
Tom Lane wrote: I don't really see a problem with doing it that way. People who want to use -W are presumably worried about the security of their local system, otherwise they would just fire up the postmaster and set a password later. No, that is exactly what I don't agree with. People

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Thu, 2004-07-15 at 15:57, Bruce Momjian wrote: We will get there --- it just seems dark at this time. Thanks for that. My comments were heartfelt, but not useful right now. I'm badly overdrawn already on my time budget, though that is my concern alone. There is more to do than I have time

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Devrim GUNDUZ
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Simon, On Thu, 15 Jul 2004, Simon Riggs wrote: We will get there --- it just seems dark at this time. Thanks for that. My comments were heartfelt, but not useful right now. I'm badly overdrawn already on my time budget, though that is my

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Thu, 2004-07-15 at 13:16, HISADAMasaki wrote: Dear Simon, I've just tested pitr_v5_2.patch and got an error message during archiving process as follows. -- begin LOG: archive command=cp /usr/local/pgsql/data/pg_xlog/ /tmp,return code=-1 -- end The command called

Re: [HACKERS] [PATCHES] serverlog rotation/functions

2004-07-15 Thread Bruce Momjian
Andreas Pflug wrote: You do something that splits the value into directory name and file name and removes every letter after %. /var/log postgresql.log.%-%-%_%%% Another idea is to allow filename wildcards in the listing so it becomes: SELECT * FROM

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Thu, 2004-07-15 at 23:18, Devrim GUNDUZ wrote: Thanks for the vote of confidence, on or off list. too many people spend a lot of money for proprietary databases, just for some missing features in PostgreSQL Agreed - PITR isn't aimed at existing users of PostgreSQL. If you use it

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Alvaro Herrera
On Thu, Jul 15, 2004 at 11:44:02PM +0100, Simon Riggs wrote: On Thu, 2004-07-15 at 13:16, HISADAMasaki wrote: -- line 236 --- - pgsignal(SIGCHLD, SIG_IGN); -- line 236 --- + pgsignal(SIGCHLD, SIG_DFL); I'm not sure I understand why its returned -1, though I'll take you

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Bruce Momjian
Simon Riggs wrote: On Thu, 2004-07-15 at 23:18, Devrim GUNDUZ wrote: Thanks for the vote of confidence, on or off list. too many people spend a lot of money for proprietary databases, just for some missing features in PostgreSQL Agreed - PITR isn't aimed at existing users of

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Mark Kirkwood
Simon Riggs wrote: First, thanks for sticking with it to test this. I've not received such a message myself - this is interesting. Is it possible to copy that directory to one side and re-run the test? Add another parameter in postgresql.conf called archive_debug = true Does it happen identically

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Bruce Momjian
Simon Riggs wrote: On Thu, 2004-07-15 at 15:57, Bruce Momjian wrote: We will get there --- it just seems dark at this time. Thanks for that. My comments were heartfelt, but not useful right now. I'm badly overdrawn already on my time budget, though that is my concern alone. There is

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Glen Parker
Simon Riggs wrote: On Thu, 2004-07-15 at 23:18, Devrim GUNDUZ wrote: Thanks for the vote of confidence, on or off list. too many people spend a lot of money for proprietary databases, just for some missing features in PostgreSQL Agreed - PITR isn't aimed at existing users

Re: [HACKERS] Note about robustness of transaction-related data structures

2004-07-15 Thread Alvaro Herrera
On Thu, Jul 15, 2004 at 05:33:56PM -0400, Tom Lane wrote: The underlying problem doesn't seem very reproducible --- I tried several times without seeing it again. But what I deduce from the core dump is that we had an error during subtransaction cleanup, leading to an attempt to re-abort an

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Mark Kirkwood
Couldn't agree more. Maybe we should have made more noise :-) Glen Parker wrote: Simon Riggs wrote: On Thu, 2004-07-15 at 23:18, Devrim GUNDUZ wrote: Thanks for the vote of confidence, on or off list. too many people spend a lot of money for proprietary databases, just for

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Mark Kirkwood
Simon Riggs wrote: So far: I've tried to re-create the problem as exactly as I can, but it works for me. This is clearly an important case to chase down. I assume that this is the very first time you tried recovery? Second and subsequent recoveries using the same set have a potential loophole,

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Fri, 2004-07-16 at 00:01, Alvaro Herrera wrote: On Thu, Jul 15, 2004 at 11:44:02PM +0100, Simon Riggs wrote: On Thu, 2004-07-15 at 13:16, HISADAMasaki wrote: -- line 236 --- - pgsignal(SIGCHLD, SIG_IGN); -- line 236 --- + pgsignal(SIGCHLD, SIG_DFL); I'm not sure I

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Simon Riggs
On Fri, 2004-07-16 at 00:46, Mark Kirkwood wrote: By way of contrast, using the *same* procedure (1-11), but generating 2 logs worth of INSERTS/UPDATES using 10 concurrent process *works fine* - e.g : Great...at least we have shown that something works (or can work) and have begun to

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: It occurs to me that with a check this thorough, we might be able to finesse the problem on Windows with the system returning very nonstandard timezone abbreviations. It does *not* pick up my timezone. Drat. I assume from your domain name that

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Bruce Momjian
Tom Lane wrote: I thought about restricting the scope of the TZ testing to start in 1990 or so to avoid this, but that seems certain to fall foul of the other problem, which is distinguishing closely-related timezones (cf Chris K-L discovering that he lives in Antarctica, a few days back...)

Re: [HACKERS] Is trust really a good default?

2004-07-15 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: There are of course some questions about how to document this effectively, so that it doesn't create more confusion than it avoids. Yes, that is another thing I'm afraid of. Yeah, if you look up-thread you'll find me expressing the

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Oliver Jowett
Tom Lane wrote: I thought about restricting the scope of the TZ testing to start in 1990 or so to avoid this, but that seems certain to fall foul of the other problem, which is distinguishing closely-related timezones (cf Chris K-L discovering that he lives in Antarctica, a few days back...) How

[HACKERS] pg_dump - option for just dumping sequences?

2004-07-15 Thread Justin Clift
Hi all, Came across a scenario today where I want to backup the latest values of sequences in the database, so they can be restored along with the data from a COPY command. Looked at pg_dump's output options (for 7.4/7.5) and there doesn't seem to be an easy way for just dumping sequence

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2004-07-16 at 00:01, Alvaro Herrera wrote: My manpage for signal(2) says that you shouldn't assign SIG_IGN to SIGCHLD, according to POSIX. So - I should be setting this to SIG_DFL and thats good for everyone? Yeah, we learned the same lesson in

[HACKERS] Converting epoch to timestamp

2004-07-15 Thread Michael Glaesemann
Hello all. In IRC, one of the questions we get from time to time is how to convert UNIX epoch to PostgreSQL timestamp. Users are often surprised there isn't a builtin cast or function that does this. I've put together two simple SQL functions that accomplish this. I think they would make the

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Tom Lane
Oliver Jowett [EMAIL PROTECTED] writes: How about scanning backwards until you have = 1 choice or decide to give up? Hmm ... that really seems like not a bad idea. Scan all the available timezones, score each on how far back it goes before a mismatch, take the one that goes furthest back.

Re: [HACKERS] How to display privileges in psql

2004-07-15 Thread Christopher Kings-Lynne
Maybe other people want to comment on this issue. So far, \z and \dp have shown privileges of tables. Now \dn+ and \db+ show the privileges of schemas and tablespaces. Should we, for consistency, move the table privilege display to \dt+, or should we move the schema and tablespace display to, say

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Dann Corbit
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane Sent: Thursday, July 15, 2004 9:13 PM To: Oliver Jowett Cc: Magnus Hagander; Hackers; [EMAIL PROTECTED] Subject: Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone Oliver Jowett

Re: [HACKERS] Point in Time Recovery

2004-07-15 Thread Christopher Kings-Lynne
Thanks for that. My comments were heartfelt, but not useful right now. Hi Simon, I'm sorry if I gave the impression that I thought your work wasn't worthwhile, it is :( I'm badly overdrawn already on my time budget, though that is my concern alone. There is more to do than I have time for.

Re: [HACKERS] Converting epoch to timestamp

2004-07-15 Thread Tom Lane
Michael Glaesemann [EMAIL PROTECTED] writes: create or replace function epoch_to_timestamp( integer ) returns timestamp(0) language sql as ' SELECT ''epoch''::timestamp + $1 * ''1 second''::interval; '; This is in fact wrong, unless you live in the British Isles:

Re: [HACKERS] pg_dump - option for just dumping sequences?

2004-07-15 Thread Christopher Kings-Lynne
Does anyone else think it would be a useful thing adding an option to pg_dump for outputting just the sequences? Or perhaps there's an easy way of doing this I just haven't thought of? Can't you just grep the output for pg_catalog.setval and SET SESSION AUTHORIZATION? Chris

Re: [pgsql-hackers-win32] [HACKERS] Weird new time zone

2004-07-15 Thread Tom Lane
Dann Corbit [EMAIL PROTECTED] writes: All translations between UTC time and local time are based on the following formula: UTC = local time + bias Surely not. Or has Windows not heard of daylight-savings time? Or perhaps they have, but are not aware that the DST laws have changed often in

Re: [HACKERS] Converting epoch to timestamp

2004-07-15 Thread Christopher Kings-Lynne
This is in fact wrong, unless you live in the British Isles: the result will be off by your timezone displacement from UTC. Correct is to use timestamptz not timestamp. As an example: timestamp 1089953023 equates to Fri Jul 16 2004, 00:43:43 EDT according to strftime() on my machine (I live in US

Re: [HACKERS] pg_dump - option for just dumping sequences?

2004-07-15 Thread Justin Clift
Christopher Kings-Lynne wrote: Does anyone else think it would be a useful thing adding an option to pg_dump for outputting just the sequences? Or perhaps there's an easy way of doing this I just haven't thought of? Can't you just grep the output for pg_catalog.setval and SET SESSION

Re: [HACKERS] Converting epoch to timestamp

2004-07-15 Thread Tom Lane
Christopher Kings-Lynne [EMAIL PROTECTED] writes: I would actually prefer this syntax: SELECT EXTRACT(EPOCH FROM TIMESTAMP '2003-01-01 11:23:44'); Yeah, but I think Michael's question had to do with going the other way (numeric to timestamp). regards, tom lane

[HACKERS] XLogWrite: write request 0/53A4000 is past end of log 0/53A4000

2004-07-15 Thread lists
i had a lot of debugging enabled. i went to create an index, and pgsql dumped on me. i tried restarting several times - no luck. starting postgresql gives: LOG: database system shutdown was interrupted at 2004-07-16 01:11:56 EDT LOG: checkpoint record is at 0/53A3FC0 LOG: redo record is at

Re: [HACKERS] Converting epoch to timestamp

2004-07-15 Thread Michael Glaesemann
On Jul 16, 2004, at 1:55 PM, Tom Lane wrote: This is in fact wrong, unless you live in the British Isles: the result will be off by your timezone displacement from UTC. Correct is to use timestamptz not timestamp. Thanks. Timestamps and time zones are a challenge for me. A cast from integer is