Re: [HACKERS] COPY is not working
On Fri, Apr 30, 2010 at 12:56 AM, Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Jaime Casanova jcasa...@systemguards.com.ec wrote: COPY is not working on latest HEAD? regression=# select * from a; aa 32 56 (2 rows) regression=# COPY a TO '/tmp/copy_test'; COPY 0 -- Please send the actual test pattern and your environment information to reproduce the misbehavior. It works fine on my machine. it's the regression database generated by make installcheck, there is an a table that is parent table (it has children), seems like COPY is not getting the data from the inherited tables but only from the parent -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY is not working
Jaime Casanova jcasa...@systemguards.com.ec wrote: ah! this is because COPY doesn't follow inherited tables... should it? Yes. You can use COPY (SELECT * FROM a) TO instead to copy all tuples. http://developer.postgresql.org/pgdocs/postgres/sql-copy.html | COPY can only be used with plain tables, not with views. | However, you can write COPY (SELECT * FROM viewname) TO Should we add or parent tables after not with views? To be exact, it would be 'COPY a parent table TO' only copies tuples in the parent table and does not copy inherited child tables. regression=# CREATE TABLE a (aa integer); CREATE TABLE regression=# CREATE TABLE b () INHERITS (a); CREATE TABLE regression=# INSERT INTO b VALUES(32), (56); INSERT 0 2 regression=# select * from a; aa 32 56 (2 rows) regression=# COPY a TO '/tmp/copy_test'; COPY 0 regression=# COPY (SELECT * FROM a) TO '/tmp/copy_test'; COPY 2 Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY is not working
On Fri, Apr 30, 2010 at 1:13 AM, Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Jaime Casanova jcasa...@systemguards.com.ec wrote: ah! this is because COPY doesn't follow inherited tables... should it? Yes. You can use COPY (SELECT * FROM a) TO instead to copy all tuples. http://developer.postgresql.org/pgdocs/postgres/sql-copy.html | COPY can only be used with plain tables, not with views. | However, you can write COPY (SELECT * FROM viewname) TO Should we add or parent tables after not with views? To be exact, it would be 'COPY a parent table TO' only copies tuples in the parent table and does not copy inherited child tables. +1 on make this clear on the docs -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] failed assertion and panic in standby mode
On Fri, Apr 30, 2010 at 12:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jaime Casanova jcasa...@systemguards.com.ec writes: i was trying recent HS and get this when trying to start the standby, TRAP: FailedAssertion(!(( (metabuffer) != 0 (metabuffer) = -NLocBuffer (metabuffer) = NBuffers )), File: ginxlog.c, Line: 590) Hm, can you provide a test case? i wasn't able to reproduce the assertion, although i'm pretty sure what i did when i got that... it's late for me now, tomorrow i will try again and will post the exact steps -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] failed assertion and panic in standby mode
Jaime Casanova wrote: i was trying recent HS and get this when trying to start the standby, actually i was expecting a crash because i use full_page_writes=off and i guess it won't work. Maybe we could say full_page_writes=off and wal_level=hot_standby are conflicting and avoid such setup? It's supposed to work. PANIC: btree_redo: unknown op code 208 CONTEXT: xlog redo UNKNOWN Hmm, I see a bug in btree_redo handling of XLOG_BTREE_REUSE_PAGE. If hot standby is not enabled in the standby, it chokes on that record type. It's missing from btree_desc too. Fixed. This demonstrates that there really has been little testing of warm standby with hot standby disabled :-). -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
On tor, 2010-04-29 at 17:34 -0400, Bruce Momjian wrote: I talked to a few people personally about this, and it seems there was a misunderstanding. I was not asking if pg_migrator should be in 9.0 beta1. I was asking if we should think about putting it into a later 9.0 beta, like 9.0 beta3. It would be another major 9.0 feature. If it's supposed to be a major feature, then it should be in src/bin, and fully integrated in the install, the documentation, etc. If you want to put it in contrib because the standards are more lax there, then by definition it's not really a major feature, it's just a random feature that was snuck in. You can't have it both ways. My personal feeling is that pg_migrator should be fully integrated, but it's too late for that, obviously. Let's do it for 9.1. I also think that the standards for contrib should not be so lax that a completely new module can be added after beta. (This is mostly informed by the feeling that contrib should go away entirely.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Invalidating dependent views and functions
I've been working on a brute force method of saving view and function source when changes to an underlying object force that object to be dropped. But I think there is a way for Postgres to handle this that wouldn't be too hard to implement and would be extremely useful for us users. Problem: We need to change the last_name column of the people table from varchar(30) to varchar(50). You issue the alter table command only to be reminded that the people table is quite popular. It will likely be used in dozens of views and many of those views will have dependent views, and lets say there are some dependent functions too. And you have to drop all of them if you want to alter your column. Once they are dropped you can alter your column and then start digging through your source code repository to rebuild all of those views and functions that you just dropped. Proposal: Add an invalid flag to pg_class. Invalid objects would be ignored when doing dependency checks for DDL statements. And an exception would be thrown when an invalid object is called. This is similar to what Oracle does. And most Oracle tools have find and compile invalid objects with a statement like: ALTER VIEW foo RECOMPILE; ALTER PACKAGE bar RECOMPILE BODY; Oracle invalidates objects without warning. But maybe we could keep the current behavior and add an invalidate option. ALTER TABLE people ALTER last_name VARCHAR(50); -- Throw exception can not alter table with dependents ALTER TABLE people ALTER last_name VARCHAR(50) INVALIDATE; -- Alters column and invalidates any dependent objects Is this a viable option? Scott Bailey -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invalidating dependent views and functions
Scott Bailey arta...@comcast.net wrote: Problem: We need to change the last_name column of the people table from varchar(30) to varchar(50). Proposal: Add an invalid flag to pg_class. Your example is one of the simplest cases, but there are other complex usages. For example, shrinking varchar length, altering indexed columns, CREATE FUNCTION RETURNS altered_table_type, and so on. Can your proposal solve all (or almost all) use-cases? I think we need to have such flag fields for each catalog tables if we support invalid status. ALTER TABLE people ALTER last_name VARCHAR(50) INVALIDATE; -- Alters column and invalidates any dependent objects IMHO, I don't like the invalid flags. If we can recompile objects later, why don't we recomple them at the same time? ALTER TABLE people ALTER last_name TYPE varchar(50) CASCADE; -- Alters column and *recompile* any dependent objects However, dependent objects are not only in the database, but also in the client applications. That's why we allow CREATE OR REPLACE VIEW only to add columns, but disallow to modify existing columns. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Peter Eisentraut pete...@gmx.net writes: My personal feeling is that pg_migrator should be fully integrated, but it's too late for that, obviously. Let's do it for 9.1. +1 I also think that the standards for contrib should not be so lax that a completely new module can be added after beta. (This is mostly informed by the feeling that contrib should go away entirely.) +1 For the record, the contrib replacement would look like proper Extension handling in dumprestore, PGXS support for windows, and PGAN for source level archive distribution. We'd still rely on distributions support for binaries. Those are the technical layers we need, then we'd have a PGAN entry for replacing contrib, and a host of other ones. The contrib Archive Network would contain -core reviewed (and maintained?) extensions, the other ones are on their own. Maybe the main other one would be (could be?) a new component of pgfoundry. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Patch for PKST timezone
Hi all, Please accept attached patch for the following problem. af...@aftab-laptop:/opt/dev/pgsql/install/dbserver/bin$ af...@aftab-laptop:/opt/dev/pgsql/install/dbserver/bin$ ./psql postgres psql (9.0beta1) Type help for help. postgres=# SHOW timezone; TimeZone -- Asia/Karachi (1 row) postgres=# postgres=# CREATE TABLE test_table (c1 INT, c2 TIMESTAMP DEFAULT timeofday()::TIMESTAMP); CREATE TABLE postgres=# INSERT INTO test_table VALUES (1); ERROR: invalid input syntax for type timestamp: Fri Apr 30 15:36:43.906075 2010 PKST postgres=# And here is a little bit information about the system I am using. af...@aftab-laptop:/opt/dev/pgsql/install/dbserver/bin$ uname -a Linux aftab-laptop 2.6.31-20-generic #58-Ubuntu SMP Fri Mar 12 05:23:09 UTC 2010 i686 GNU/Linux af...@aftab-laptop:/opt/dev/pgsql/install/dbserver/bin$ af...@aftab-laptop:/opt/dev/pgsql/install/dbserver/bin$ ./pg_config --version PostgreSQL 9.0beta1 af...@aftab-laptop:/opt/dev/pgsql/install/dbserver/bin$ Thanks, -- Aftab Hussain, EntepriseDB http://www.enterprisedb.com diff --git a/src/timezone/tznames/Asia.txt b/src/timezone/tznames/Asia.txt index 3493e0d..483d314 100644 --- a/src/timezone/tznames/Asia.txt +++ b/src/timezone/tznames/Asia.txt @@ -202,6 +202,7 @@ PETT43200# Petropavlovsk-Kamchatski Time # (Asia/Kamchatka) PHT 28800# Phillipine Time (not in zic) PKT 18000# Pakistan Time (not in zic) +PKST21600 D # Pakistan Summer Time (not in zic) QYZT21600# Kizilorda Time # (Asia/Qyzylorda) SAKST 39600 D # Sakhalin Summer Time diff --git a/src/timezone/tznames/Default b/src/timezone/tznames/Default index 522eedd..f57673e 100644 --- a/src/timezone/tznames/Default +++ b/src/timezone/tznames/Default @@ -335,6 +335,7 @@ PETT43200# Petropavlovsk-Kamchatski Time # (Asia/Kamchatka) PHT 28800# Phillipine Time (not in zic) PKT 18000# Pakistan Time (not in zic) +PKST21600 D # Pakistan Summer Time (not in zic) SGT 28800# Singapore Time # (Asia/Singapore) TJT 18000# Tajikistan Time -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] missing file in git repo
I think postgres git repo is broken. The compilation of REL7_4_STABLE from git fails on my system with: make -C src all make -C port all make[2]: *** No rule to make target `sprompt.o', needed by `libpgport.a'. Stop. There is no sprompt.c in src/port in the sources obtained from git. However, there is one in CVS: http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/port/sprompt.c?only_with_tag=REL7_4 This looks like the initial synchronization issue, since this file is there for really long time and appears not to be touched by any commit since 2003. -- Alexey Klyukin http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
Alexey Klyukin wrote: I think postgres git repo is broken. The compilation of REL7_4_STABLE from git fails on my system with: make -C src all make -C port all make[2]: *** No rule to make target `sprompt.o', needed by `libpgport.a'. Stop. There is no sprompt.c in src/port in the sources obtained from git. However, there is one in CVS: http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/port/sprompt.c?only_with_tag=REL7_4 This looks like the initial synchronization issue, since this file is there for really long time and appears not to be touched by any commit since 2003. I don't think the git repo was ever considered working for the backbranches at all... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY is not working
2010/4/30 Jaime Casanova jcasa...@systemguards.com.ec: On Fri, Apr 30, 2010 at 1:13 AM, Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Jaime Casanova jcasa...@systemguards.com.ec wrote: ah! this is because COPY doesn't follow inherited tables... should it? Yes. You can use COPY (SELECT * FROM a) TO instead to copy all tuples. http://developer.postgresql.org/pgdocs/postgres/sql-copy.html | COPY can only be used with plain tables, not with views. | However, you can write COPY (SELECT * FROM viewname) TO Should we add or parent tables after not with views? To be exact, it would be 'COPY a parent table TO' only copies tuples in the parent table and does not copy inherited child tables. +1 on make this clear on the docs It works with parent table. Depends if the parent table have data or not. COPY is more like SELECT FROM ONLY table, isn't it ? -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Cédric Villemain -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
2010/4/30 Stefan Kaltenbrunner ste...@kaltenbrunner.cc: Alexey Klyukin wrote: I think postgres git repo is broken. The compilation of REL7_4_STABLE from git fails on my system with: make -C src all make -C port all make[2]: *** No rule to make target `sprompt.o', needed by `libpgport.a'. Stop. There is no sprompt.c in src/port in the sources obtained from git. However, there is one in CVS: http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/port/sprompt.c?only_with_tag=REL7_4 This looks like the initial synchronization issue, since this file is there for really long time and appears not to be touched by any commit since 2003. I don't think the git repo was ever considered working for the backbranches at all... Really ?! Then we have to remove the backbranches from the git. http://wiki.postgresql.org/wiki/Working_with_Git#Using_Back_Branches Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Cédric Villemain -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] failed assertion and panic in standby mode
On Fri, 2010-04-30 at 09:35 +0300, Heikki Linnakangas wrote: Hmm, I see a bug in btree_redo handling of XLOG_BTREE_REUSE_PAGE. If hot standby is not enabled in the standby, it chokes on that record type. It's missing from btree_desc too. Yep, correct fix. This demonstrates that there really has been little testing of warm standby with hot standby disabled :-). Agreed, exactly why I'm not convinced turning it off makes you safer. Most people will turn it on and so it will remain a less well tested path through the code. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
Cédric Villemain wrote: 2010/4/30 Stefan Kaltenbrunner ste...@kaltenbrunner.cc: I don't think the git repo was ever considered working for the backbranches at all... Really ?! Then we have to remove the backbranches from the git. http://wiki.postgresql.org/wiki/Working_with_Git#Using_Back_Branches Yeah, or fix them. FWIW, REL8_4_STABLE and REL8_3_STABLE are working, I just compared them against CVS checkouts. REL8_2_STABLE is not, it's missing a security-related patch from December. I didn't check the other branches. I have nothing against dropping the older branches, but please keep at least REL8_4_STABLE, I use it frequently. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invalidating dependent views and functions
On Fri, Apr 30, 2010 at 3:33 AM, Scott Bailey arta...@comcast.net wrote: Proposal: Add an invalid flag to pg_class. Invalid objects would be ignored when doing dependency checks for DDL statements. And an exception would be thrown when an invalid object is called. This is similar to what Oracle does. And most Oracle tools have find and compile invalid objects with a statement like: ALTER VIEW foo RECOMPILE; ALTER PACKAGE bar RECOMPILE BODY; Keep in mind that our implementation is apparently quite different from Oracle's. Of course I have no idea what they do under the hood, but we don't even store the original text of the view. Instead, we store a parsed version of the view text that refers to the target objects logically rather than by name. That has some advantages; for example, you can rename a column in some other table that the view uses, and nothing breaks. You can rename a whole table that is used by the view, and nothing breaks. Even if we added storage for the text of the view, recompiling it might result in some fairly astonishing behavior - you might suddenly be referring to tables or columns that were quite different from the ones you originally targeted, if the old ones were renamed out of the way and new, eponymous ones were added. I'm familiar with the view-dependency-hell problem you mention, having fought with it (succesfully, I'm pleased to say, using a big Perl script to manage things - and also - obligatory dig here - to work around our lack of support for CREATE IF NOT EXISTS) on many occasions, but I don't have any brilliant ideas about how to solve it. I would like to eventually support ALTER VIEW ... DROP COLUMN; note that we do now support ADDING columns to a view using CREATE OR REPLACE as long as all the new ones are at the end. But neither of those things is going to help with a case like yours, when you want to change the type of the column. I'm not really sure what to do about that case. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add column if not exists (CINE)
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: We can artificially make this problem as complicated as we wish, but the people who are asking for this feature (including me) will, I believe, be quite happy with a solution that throws, say, a NOTICE instead of an ERROR when the object already exists, and then returns without doing anything further. There are very few, if any, definitional issues here, except by people who are brainstorming crazy alternative behaviors whose actual usefulness I very much doubt. CREATE OR REPLACE is indeed much more complicated. In fact, for tables, I maintain that you'll need to link with -ldwim to make it work properly. This may in fact be an appropriate way to handle the case for tables, given the complexity of their definitions. However, the original point of the thread was about what to do for columns. I still say that COR rather than CINE semantics would be appropriate for columns. I have added this TODO item: Allow CREATE TABLE to optionally create a table if it does not already exist, without throwing an error The fact that tables contain data makes this more complex than other CREATE OR REPLACE operations. * http://archives.postgresql.org/pgsql-hackers/2010-04/msg01300.php -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Cédric Villemain wrote: 2010/4/30 Stefan Kaltenbrunner ste...@kaltenbrunner.cc: I don't think the git repo was ever considered working for the backbranches at all... Really ?! Then we have to remove the backbranches from the git. http://wiki.postgresql.org/wiki/Working_with_Git#Using_Back_Branches Yeah, or fix them. This thread isn't exactly leaving me with a warm fuzzy feeling about moving the master repository to git. *Why* is the mirror broken, and what assurances do we have that the problem won't recur in the proposed transition? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invalidating dependent views and functions
Scott Bailey arta...@comcast.net writes: Proposal: Add an invalid flag to pg_class. Invalid objects would be ignored when doing dependency checks for DDL statements. And an exception would be thrown when an invalid object is called. IMO, the way Oracle does this pretty much sucks, and shouldn't be emulated. If they know how to recompile the view, why don't they just do it? What you describe is about as user-unfriendly as it gets. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Dimitri Fontaine dfonta...@hi-media.com writes: Peter Eisentraut pete...@gmx.net writes: I also think that the standards for contrib should not be so lax that a completely new module can be added after beta. (This is mostly informed by the feeling that contrib should go away entirely.) +1 For the record, the contrib replacement would look like proper Extension handling in dumprestore, PGXS support for windows, and PGAN for source level archive distribution. We'd still rely on distributions support for binaries. Both of you are living in some fantasy land. The reason contrib is held to a lower standard than core is that nobody is willing to put the same level of effort into contrib. There are modules in there (most of them, in fact) that haven't been touched for years, other than as part of system-wide search-and-replace patches. Extension support is not going to magically fix that and cause maintenance effort to appear from nowhere. In the end, the main useful function that contrib serves is to provide examples of how to write Postgres extensions. Because of that, removing it as Peter suggests doesn't seem like a good idea to me. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Cédric Villemain wrote: 2010/4/30 Stefan Kaltenbrunner ste...@kaltenbrunner.cc: I don't think the git repo was ever considered working for the backbranches at all... Really ?! Then we have to remove the backbranches from the git. http://wiki.postgresql.org/wiki/Working_with_Git#Using_Back_Branches Yeah, or fix them. This thread isn't exactly leaving me with a warm fuzzy feeling about moving the master repository to git. *Why* is the mirror broken, and what assurances do we have that the problem won't recur in the proposed transition? AFAICT it is broken because the particular tool that is used, which is the only one that supports an incremental mode, is a bit broken. I am told that the non-incremental tools are more robust. That said, this is more than a little annoying. It means, for example, that I can't test out a Git mode for the buildfarm client on all the back branches. If any Ruby hacker feels like fixing it please speak up. The reported source of the software seems to have gone away. I can let you have my copy, which reliably reproduces the error, so we have a good failure test case. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Tom Lane wrote: Dimitri Fontaine dfonta...@hi-media.com writes: Peter Eisentraut pete...@gmx.net writes: I also think that the standards for contrib should not be so lax that a completely new module can be added after beta. (This is mostly informed by the feeling that contrib should go away entirely.) +1 For the record, the contrib replacement would look like proper Extension handling in dumprestore, PGXS support for windows, and PGAN for source level archive distribution. We'd still rely on distributions support for binaries. Both of you are living in some fantasy land. The reason contrib is held to a lower standard than core is that nobody is willing to put the same level of effort into contrib. There are modules in there (most of them, in fact) that haven't been touched for years, other than as part of system-wide search-and-replace patches. Extension support is not going to magically fix that and cause maintenance effort to appear from nowhere. In the end, the main useful function that contrib serves is to provide examples of how to write Postgres extensions. Because of that, removing it as Peter suggests doesn't seem like a good idea to me. Quite so. Getting a better extensions mechanism doesn't mean we should abandon what we currently have, IMNSHO. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
On fre, 2010-04-30 at 10:45 -0400, Tom Lane wrote: In the end, the main useful function that contrib serves is to provide examples of how to write Postgres extensions. Maybe, but pg_migrator surely doesn't fit that. And neither does about a third of the other contrib modules, IMO. Because of that, removing it as Peter suggests doesn't seem like a good idea to me. contrib means many things to many people, and that's exactly the problem in my mind: It doesn't mean anything in particular. If we were to separate it into - examples - production-quality add-ons with small user base - production-quality add-ons that everyone wants, but we keep them as plugins because plugins are cool - experimental code that we wanted to ship anyway - (historically) differently licensed code then these discussions would be much simpler. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
Andrew Dunstan and...@dunslane.net wrote: If any Ruby hacker feels like fixing it please speak up. I can't take it on any time soon. If nobody else picks it up, I can get to it eventually. Anyone taking it on might want to read through the thread which starts at: http://archives.postgresql.org/pgsql-hackers/2010-01/msg01679.php I reviewed the code enough to be pretty convinced that the problem could occur if a file was committed after a scan for changes had passed the file in question, and before it hit the end of the scan. The reported source of the software seems to have gone away. I can let you have my copy, which reliably reproduces the error, so we have a good failure test case. If it's not as recent as this: http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/log/132 we might want to bring in (or at least review) what's been committed since we grabbed our copy. Some of the fixes mentioned sound like they might possibly be related. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invalidating dependent views and functions
On Fri, Apr 30, 2010 at 8:08 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Apr 30, 2010 at 3:33 AM, Scott Bailey arta...@comcast.net wrote: Proposal: Add an invalid flag to pg_class. Invalid objects would be ignored when doing dependency checks for DDL statements. And an exception would be thrown when an invalid object is called. This is similar to what Oracle does. And most Oracle tools have find and compile invalid objects with a statement like: ALTER VIEW foo RECOMPILE; ALTER PACKAGE bar RECOMPILE BODY; Keep in mind that our implementation is apparently quite different from Oracle's. Of course I have no idea what they do under the hood, but we don't even store the original text of the view. Instead, we store a parsed version of the view text that refers to the target objects logically rather than by name. That has some advantages; for example, you can rename a column in some other table that the view uses, and nothing breaks. You can rename a whole table that is used by the view, and nothing breaks. Even if we added storage for the text of the view, recompiling it might result in some fairly astonishing behavior - you might suddenly be referring to tables or columns that were quite different from the ones you originally targeted, if the old ones were renamed out of the way and new, eponymous ones were added. I'm familiar with the view-dependency-hell problem you mention, having fought with it (succesfully, I'm pleased to say, using a big Perl script to manage things - and also - obligatory dig here - to work around our lack of support for CREATE IF NOT EXISTS) on many occasions, but I don't have any brilliant ideas about how to solve it. I would like to eventually support ALTER VIEW ... DROP COLUMN; note that we do now support ADDING columns to a view using CREATE OR REPLACE as long as all the new ones are at the end. But neither of those things is going to help with a case like yours, when you want to change the type of the column. I'm not really sure what to do about that case. We discussed keeping view sources for invalidation purposes in depth earlier. The main takeaway was that recompiling view sources simply doesn't work: if your view definition is: 'select * from table', the recompile would add fields to the view which SQL (unfortunately) expressly forbids. This is maybe solvable, but complicated. aside: I've been lobbying for (somefoo).* to NOT do this, that is, that is allow it to pick up extra fields on somefoo as they appear, with not so great results so far. I happen to think that the way functions are invalidated right now based on table changes actually work pretty well. Plans are invalidated appropriately and functions are dropped if you suffer major argument changes. Before thinking about improving this, you have to grapple with (for starters) the mess of interactions with search_path and function definitions. IOW, functions not getting planned until they are used is a nice property. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: WAL page magic number (was Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct)
Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Browsing the history of XLOG_PAGE_MAGIC, it used to be incremented by one whenever the format changes, at least as long as it has been been in xlog_internal.h. It started at 0xD05B in 2005, and was incremented by one by each commit till 0xD062. But then the hot standby patch bumped it to 0xD166, and on March 19th Simon changed it to 0x9002, and on March 28th to 0x9003. Is there a plan somewhere on all that, or was it just random whacking? Random whacking. Simon seems to have decided to try to make the number mean something, but it never meant anything before. It's not a version number, it's a magic identifier. We might need to avoid certain values to prevent collisions with other file formats, so I'm suspicious of trying to make it match up with some expectation. Ok, for better or worse, I whacked it back to the old series. Agreed. Making it match the version would be odd because you then have t bump it for every major release. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? If that's what you want to do, use archive_mode. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Robert Haas wrote: On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. ?Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. ?We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? If that's what you want to do, use archive_mode. Uh, I assume that will require me to store the WAL files somewhere else, rather than keeping them in /pg_xlog, which I thought was the goal. Am I missing something? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing file in git repo
Kevin Grittner wrote: The reported source of the software seems to have gone away. I can let you have my copy, which reliably reproduces the error, so we have a good failure test case. If it's not as recent as this: http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/log/132 we might want to bring in (or at least review) what's been committed since we grabbed our copy. Some of the fixes mentioned sound like they might possibly be related. OK, there is good news on this front. When last I looked this repo was giving errors, so I worked from the copy of the code I already had. (So thanks to Kevin for prodding me to check again.) With the latest fromcvs and rcsparse packages downloaded from this site I can now make and run the 7.4 and 8.1 branches from the imported repository (I haven't checked other branches yet.) So we need to look at how hard it would be to upgrade the scripts on git.postgresql.org, and if by doing so we can fix its versions of the back branches. It would also be a good idea to have a script that regularly checks out the important branches from git and checks that they are identical to what is in CVS. That might help to increase Tom's confidence level. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Simon Riggs wrote: On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. LOL, good one. I assume it was done so it would start with 'wal', but I see 'max_wal_senders', which doesn't start with 'wal' and would match your suggestion exactly. I think we should either rename 'wal_keep_segments' or 'max_wal_senders'. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, Apr 30, 2010 at 1:39 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. ?Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. ?We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? If that's what you want to do, use archive_mode. Uh, I assume that will require me to store the WAL files somewhere else, rather than keeping them in /pg_xlog, which I thought was the goal. Am I missing something? Well, one of us is. Why would you want to retain all of your WAL logs in pg_xlog forever? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On 04/30/2010 01:53 PM, Robert Haas wrote: Well, one of us is. Why would you want to retain all of your WAL logs in pg_xlog forever? ...Robert To create or re-synchronize SR slaves, one could change wal_keep_segments to -1, run a backup, wait for the slaves to catch up, and change it back to the default. This way no segments would be deleted until the system has reached a stable state. -- m. tharp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Robert Haas wrote: On Fri, Apr 30, 2010 at 1:39 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. ?Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. ?We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? If that's what you want to do, use archive_mode. Uh, I assume that will require me to store the WAL files somewhere else, rather than keeping them in /pg_xlog, which I thought was the goal. ?Am I missing something? Well, one of us is. Why would you want to retain all of your WAL logs in pg_xlog forever? Well, this email thread mentioned a case where you needed to increase wal_keep_segments to a sufficiently-high value, and of course figuring out such a value is harder than just having a way of turning off recycling with -1. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, 2010-04-30 at 13:52 -0400, Robert Haas wrote: On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? I see the thinking, but why would you ever set it to be something that is *less* than the existing numbers? That would be pointless and indeed, does nothing. The only time you touch it at all is when you set it to be a value higher than the number of files that would normally be kept, and when that is the case it *will* be the maximum. So I say, max_wal_segments = 0 (default) meaning no limit, we just rotate as needed. We put a comment in the docs to say that if a value is selected less than 2*checkpoint_segments+1 then the value is overridden. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, 2010-04-30 at 13:58 -0400, Bruce Momjian wrote: Robert Haas wrote: On Fri, Apr 30, 2010 at 1:39 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. ?Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. ?We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? If that's what you want to do, use archive_mode. Uh, I assume that will require me to store the WAL files somewhere else, rather than keeping them in /pg_xlog, which I thought was the goal. ?Am I missing something? Well, one of us is. Why would you want to retain all of your WAL logs in pg_xlog forever? Well, this email thread mentioned a case where you needed to increase wal_keep_segments to a sufficiently-high value, and of course figuring out such a value is harder than just having a way of turning off recycling with -1. I think the only sensible setting is as big as my (available) disk space. Any higher and you're going to crash, any lower and you'll invalidate your backup for no reason. -1 emulates current behaviour, BTW Still think we should rename it, in which case 0 is same as no maximum. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Andrew Dunstan and...@dunslane.net writes: Quite so. Getting a better extensions mechanism doesn't mean we should abandon what we currently have, IMNSHO. Yeah, agreed. Exactly what I proposed. The only change is the distribution mean. Either we keep things as they are now exactly, or we use the new Archive Network facilities, in the spirit of being sure they get exercised, requiring that commiters gets to use them and maybe use a separate code repository for contribs. Or simply an adjusted Makefile. Summary: the only proposed change is how to do it, not what we do. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Robert Haas wrote: On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? Yeah, min_wal_segments or something would make sense. It sounds about as good or bad as wal_keep_segments to me. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Bruce Momjian wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Umm, you can't keep all segments around forever, can you? Surely you have to recycle them sooner or later or you will run out of disk space. I guess you could move that responsibility to a user-written script, but we haven't traditionally encouraged or supported people to mess with the contents of pg_xlog. That would require some more thinking IMHO, not 9.0 material. In practice, you can just set wal_keep_segments to some ridiculously high value to achieve the same result. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Yeah, min_wal_segments or something would make sense. Surely it would confuse people to see they have fewer than min_wal_segments WAL segments. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Heikki Linnakangas wrote: Robert Haas wrote: On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? Yeah, min_wal_segments or something would make sense. It sounds about as good or bad as wal_keep_segments to me. I admit I never liked keep but couldn't think of better wording. I do like the proposed wording better. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: WAL page magic number (was Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct)
Bruce Momjian wrote: Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Browsing the history of XLOG_PAGE_MAGIC, it used to be incremented by one whenever the format changes, at least as long as it has been been in xlog_internal.h. It started at 0xD05B in 2005, and was incremented by one by each commit till 0xD062. But then the hot standby patch bumped it to 0xD166, and on March 19th Simon changed it to 0x9002, and on March 28th to 0x9003. Is there a plan somewhere on all that, or was it just random whacking? Random whacking. Simon seems to have decided to try to make the number mean something, but it never meant anything before. It's not a version number, it's a magic identifier. We might need to avoid certain values to prevent collisions with other file formats, so I'm suspicious of trying to make it match up with some expectation. Ok, for better or worse, I whacked it back to the old series. Agreed. Making it match the version would be odd because you then have t bump it for every major release. Not necessarily. It would make sense to me to just keep it the same if there has been no changes. The magic number would indicate the first version that used that format. I'm not advocating a switch to that system, but it would be sensible as well. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Robert Haas robertmh...@gmail.com writes: On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? Indeed. It would really be more like min_wal_segments, if we wanted to name it that way. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Heikki Linnakangas wrote: Bruce Momjian wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. I wasn't really thinking of this use case, but you could set wal_keep_segments high enough. Ah. Okay, that seems like a workable approach, at least for people with reasonably predictable WAL loads. We could certainly improve on it later to make it more bulletproof, but it's usable now --- if we relax the error checks. (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Umm, you can't keep all segments around forever, can you? Surely you have to recycle them sooner or later or you will run out of disk space. I guess you could move that responsibility to a user-written script, but we haven't traditionally encouraged or supported people to mess with the contents of pg_xlog. That would require some more thinking IMHO, not 9.0 material. In practice, you can just set wal_keep_segments to some ridiculously high value to achieve the same result. Which is where my 'wal_keep_segments = -1' idea came from. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, Apr 30, 2010 at 2:08 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-04-30 at 13:52 -0400, Robert Haas wrote: On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean keep all segments? Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? I see the thinking, but why would you ever set it to be something that is *less* than the existing numbers? That would be pointless and indeed, does nothing. The only time you touch it at all is when you set it to be a value higher than the number of files that would normally be kept, and when that is the case it *will* be the maximum. So I say, max_wal_segments = 0 (default) meaning no limit, we just rotate as needed. We put a comment in the docs to say that if a value is selected less than 2*checkpoint_segments+1 then the value is overridden. As you were quick to point out to me earlier this week, I am not an expert on our write-ahead logging system; however, I think you are mistaken. Perhaps Heikki could speak to the point more definitively, but I believe that the number of segments that the system retains for WAL archiving or crash recovery is variable. The purpose of this variable is to put a floor under the number of segments that are retained so that SR slaves can catch up if they fall behind. Of course, if archiving is configured, they can do that anyway using restore_command, but you might be running SR without archiving, or you might just want to set this to a small value so that the slaves don't have to keep switching between SR and archive recovery if segments get archived or checkpointed away at inconvenient times. It doesn't make a whole lot of sense to set the floor on the number of segments retained to positive infinity, except in one specific case: archiving is disabled, and you're trying to hang on to enough segments in pg_xlog to take a hot backup. As Tom said, it would be nice to have a more elegant solution to that problem, but we can do that in a future release; it's not really the primary purpose of wal_keep_segments, anyway. It certainly would not be a good idea to make the default configuration retain all WAL forever. If you did that, a user who sets up PostgreSQL and is not using SR or HS or hot backups will eventually and inevitably fill up their hard disk. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Bruce Momjian wrote: Should we allow -1 to mean keep all segments? Umm, you can't keep all segments around forever, can you? Surely you have to recycle them sooner or later or you will run out of disk space. You couldn't use that as a permanent setting, but it can make sense as a transient setting, rather than having to guess how much WAL you'll need to keep while setting up a new standby. In practice, you can just set wal_keep_segments to some ridiculously high value to achieve the same result. True. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Bruce Momjian escribió: Which is where my 'wal_keep_segments = -1' idea came from. Are you suggesting that -1 should mean keep all segments that fit on disk, but if creating a new segment fails with ENOSPC, recycle the oldest one? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Michael Tharp wrote: On 04/30/2010 01:53 PM, Robert Haas wrote: Well, one of us is. Why would you want to retain all of your WAL logs in pg_xlog forever? To create or re-synchronize SR slaves, one could change wal_keep_segments to -1, run a backup, wait for the slaves to catch up, and change it back to the default. This way no segments would be deleted until the system has reached a stable state. A slave can fall behind at any time, though. You would have to know to set wal_keep_segments to -1 before that happens. I've been thinking that in the future (read 9.1 or above), we would have a system for registering slaves in the primary server. The primary would keep track of how far each slave is, and refrain from removing WAL segments that it knows to be still needed by a slave. On the flip-side, the master wouldn't need to keep WAL around that it knows is no longer needed by any slaves. If someone has the energy, it would be possible to write a stand-alone application to do that too. It could serve old WAL files from the archive and rely recent ones from the real master. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Alvaro Herrera alvhe...@commandprompt.com writes: Bruce Momjian escribió: Which is where my 'wal_keep_segments = -1' idea came from. Are you suggesting that -1 should mean keep all segments that fit on disk, but if creating a new segment fails with ENOSPC, recycle the oldest one? No, keep means keep. Even if there were some arguable use for keep if you can, a scheme like that would render the machine unusable --- everything else on the same filesystem would be falling over. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, 2010-04-30 at 14:42 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs si...@2ndquadrant.com wrote: Why is that not called max_wal_segments? wal_keep_segments sounds like its been through Google translate. Because it's not a maximum? Indeed. It would really be more like min_wal_segments, if we wanted to name it that way. Yeh, agreed: min_wal_segments. I realised while having dinner it was the opposite, so I'm pleased everybody else got there at same time. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Kevin Grittner wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Yeah, min_wal_segments or something would make sense. Surely it would confuse people to see they have fewer than min_wal_segments WAL segments. Umm, they wouldn't see that, that's the point of the setting. The segments are not removed/recycled until there is min_wal_segments segments in pg_xlog. Except in the beginning when you set or increase the setting, when there isn't that many segments generated yet. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Kevin Grittner wrote: Surely it would confuse people to see they have fewer than min_wal_segments WAL segments. they wouldn't see that, that's the point of the setting. I was thinking, in particular, about beginners poking around to see how things look after an initdb. Perhaps that state is too transient to matter, but it struck me that you'd have fewer than the minimum at the precise time a beginner might be likely to take a look. Unless on startup (and reload?) we created min_wal_segments WAL segments if they didn't already exist. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY is not working
Jaime Casanova jcasa...@systemguards.com.ec writes: COPY is not working on latest HEAD? I've added this to the COPY notes section: para commandCOPY/command only deals with the specific table named; it does not copy data to or from child tables. Thus for example literalCOPY replaceable class=parametertable/ TO/literal shows the same data as literalSELECT * FROM ONLY replaceable class=parametertable//literal. But literalCOPY (SELECT * FROM replaceable class=parametertable/) TO .../literal can be used to dump all of the data in an inheritance hierarchy. /para regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] HS - odd process listing
While playing with HS/SR on -HEAD I noticed that setting setting max_standby_delay to -1 leads to the following kind of odd ps display: 1000 20254 9.2 11.3 8625164 5600644 ? Rs 21:22 1:50 postgres: startup process waiting for max_standby_delay (4294967295 s) This seems fairly confusing from a usability POV... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Synchronous replication patch built on SR
Please add it to the next commit-fest: https://commitfest.postgresql.org/action/commitfest_view/inprogress --- z...@cybertec.at wrote: Resending, my ISP lost my mail yesterday. :-( === Hi, attached is a patch that does $SUBJECT, we are submitting it for 9.1. I have updated it to today's CVS after the wal_level GUC went in. How does it work? First, the walreceiver and the walsender are now able to communicate in a duplex way on the same connection, so while COPY OUT is in progress from the primary server, the standby server is able to issue PQputCopyData() to pass the transaction IDs that were seen with XLOG_XACT_COMMIT or XLOG_XACT_PREPARE signatures. I did by adding a new protocol message type, with letter 'x' that's only acknowledged by the walsender process. The regular backend was intentionally unchanged so an SQL client gets a protocol error. A new libpq call called PQsetDuplexCopy() which sends this new message before sending START_REPLICATION. The primary makes a note of it in the walsender process' entry. I had to move the TransactionIdLatest(xid, nchildren, children) call that computes latestXid earlier in RecordTransactionCommit(), so it's in the critical section now, just before the XLogInsert(RM_XACT_ID, XLOG_XACT_COMMIT, rdata) call. Otherwise, there was a race condition between the primary and the standby server, where the standby server might have seen the XLOG_XACT_COMMIT record for some XIDs before the transaction in the primary server marked itself waiting for this XID, resulting in stuck transactions. I have added 3 new options, two GUCs in postgresql.conf and one setting in recovery.conf. These options are: 1. min_sync_replication_clients = N where N is the number of reports for a given transaction before it's released as committed synchronously. 0 means completely asynchronous, the value is maximized by the value of max_wal_senders. Anything in between 0 and max_wal_senders means different levels of partially synchronous replication. 2. strict_sync_replication = boolean where the expected number of synchronous reports from standby servers is further limited to the actual number of connected synchronous standby servers if the value of this GUC is false. This means that if no standby servers are connected yet then the replication is asynchronous and transactions are allowed to finish without waiting for synchronous reports. If the value of this GUC is true, then transactions wait until enough synchronous standbys connect and report back. 3. synchronous_slave = boolean (in recovery.conf) this instructs the standby server to tell the primary that it's a synchronous replication server and it will send the committed XIDs back to the primary. I also added a contrib module for monitoring the synchronous replication but it abuses the procarray.c code by exposing the procArray pointer which is ugly. It's either need to be abandoned or moved to core if or when this code is discussed enough. :-) Best regards, Zolt?n B?sz?rm?nyi [ Attachment, skipping... ] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HS - odd process listing
Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: While playing with HS/SR on -HEAD I noticed that setting setting max_standby_delay to -1 leads to the following kind of odd ps display: 1000 20254 9.2 11.3 8625164 5600644 ? Rs 21:22 1:50 postgres: startup process waiting for max_standby_delay (4294967295 s) I noticed that a lot of the HS error messages use %u to print integer parameters. This seemed like a dubious idea to me, but I hadn't made a concerted effort to get rid of it. Seems like that is what we ought to do though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Tom Lane wrote: Dimitri Fontaine dfonta...@hi-media.com writes: Peter Eisentraut pete...@gmx.net writes: I also think that the standards for contrib should not be so lax that a completely new module can be added after beta. (This is mostly informed by the feeling that contrib should go away entirely.) +1 For the record, the contrib replacement would look like proper Extension handling in dumprestore, PGXS support for windows, and PGAN for source level archive distribution. We'd still rely on distributions support for binaries. Both of you are living in some fantasy land. The reason contrib is held to a lower standard than core is that nobody is willing to put the same level of effort into contrib. There are modules in there (most of them, in fact) that haven't been touched for years, other than as part of system-wide search-and-replace patches. Extension support is not going to magically fix that and cause maintenance effort to appear from nowhere. In the end, the main useful function that contrib serves is to provide examples of how to write Postgres extensions. Because of that, removing it as Peter suggests doesn't seem like a good idea to me. So what do people want to do with pg_migrator? I don't think calling pg_migrator a major features requires it to be in /bin. We added full text search to /contrib years ago and that was a major feature. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
