Re: [HACKERS] Status report on writeable CTEs
2010/7/13 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi: On 7/12/10 9:34 PM +0300, Tom Lane wrote: Marko Tiikkajamarko.tiikk...@cs.helsinki.fi writes: ... So what I'm now thinking of is making the planner plan that as a single Query, and make the planner expand it into multiple PlannedStmts if necessary. This would break the existing planner hooks, but I don't think that's a huge problem. Does anyone see any obvious flaws in this? How will that interact with the existing rewriter? It sounds a lot like a recipe for duplicating functionality ... I was thinking that the rewriter would look at the top-level CTEs and rewrite all non-SELECT queries into a list of Queries and store that list into the CommonTableExprs. The planner would then use this information and also expand the rewrite products. Why didn't you choose this strategy? ISTM changing signature of planner() and standard_planner() makes it more difficult to commit this feature unnecessarily. I thought at first that it is possible and better to split the statement into multiple Querys in the rewriter and pass them to the planner, but as you mentioned they should collaborate each other like in ctelevelsup, so to pass a single Query to the planner is probably the answer. However, you can store the sub (ie, in WITH clause) PlannedStmt in the top level PlannedStmt and extract the sub statements in pg_plan_queries() or anywhere in the PortalXXX() then add them to the execution list. Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
2010/7/21 David Fetter da...@fetter.org: On Sat, Jul 17, 2010 at 01:15:22AM +0900, Hitoshi Harada wrote: 2010/7/17 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi: On 7/16/10 6:15 PM +0300, Hitoshi Harada wrote: 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode is exsiting one that work with single tuplestore, it might be sane to modify this so that it accepts tuplestore from Query instead of its child node. I thought about this, but I don't necessarily like the idea of overloading executor nodes. Neither do I have good shape for this solution. Maybe it's not good idea. But my concern is adding DtScanNode, which looks similar to MaterialNode. Of course each purpose is different, but quite big part will overlap each other, I think. Any ideas as to how to factor this out? Handiest ideas would be in the form of a patch ;) Yeah, that would be handiest, but I think I must wait for his first compilable patch to modify to try the idea. Current version looks quite messy and can't build. 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? It's just deconstructing queries like: WITH t AS (INSERT INTO x ... RETURING *) SELECT * FROM t; to CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *; SELECT * FROM t; While the second statement is not implemented yet, it will be so simpler. So CTAS would get expanded into CTA[row-returning query] ? Interesting. How much work would that part be? FWIW, this is getting interesting to me these days, and I think this can be separated from wCTE As hackers say, the first to try should be Marko's first design that use the list of tuplestore and DTScanNode. So if he has solid image to reach there, we can wait for him to complete his first compilable version. Then let's take it back and forth. Is it possible? And I concern we might not have concrete consensus about list of features in document form. Does it accept Recursive query? What if x refers to y that refers to x cyclicly? An external design sometimes fix the internal design, and it helps people to review the implementation. If I missed something please point me to the link. Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On 8/3/2010 7:30 PM, Hitoshi Harada wrote: As hackers say, the first to try should be Marko's first design that use the list of tuplestore and DTScanNode. So if he has solid image to reach there, we can wait for him to complete his first compilable version. Then let's take it back and forth. Is it possible? I am currently working on a version that treats all WITH queries like wCTEs. My progress can be seen in my git repo [1], branch wcte. In its current form, the patch compiles and passes all applicable regression tests but it's still very messy. I'm going to send a cleaner WIP patch to the list the minute I have one, but anyone's free to look at the git repo (and even work on it if they want!). And I concern we might not have concrete consensus about list of features in document form. Does it accept Recursive query? What if x refers to y that refers to x cyclicly? An external design sometimes fix the internal design, and it helps people to review the implementation. If I missed something please point me to the link. A recursive query should be fine as long as 1) it's SELECT-only and 2) it doesn't loop forever. A wCTE can of course refer to the result of the recursive SELECT query with INSERT .. SELECT, UPDATE .. FROM or DELETE .. USING. Cyclic dependencies are out of the scope of this patch; I'm not planning on adding any new features to regular CTEs. [1] http://git.postgresql.org/gitweb?p=users/johto/postgres.git;a=summary Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On Sat, Jul 17, 2010 at 01:15:22AM +0900, Hitoshi Harada wrote: 2010/7/17 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi: On 7/16/10 6:15 PM +0300, Hitoshi Harada wrote: 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode is exsiting one that work with single tuplestore, it might be sane to modify this so that it accepts tuplestore from Query instead of its child node. I thought about this, but I don't necessarily like the idea of overloading executor nodes. Neither do I have good shape for this solution. Maybe it's not good idea. But my concern is adding DtScanNode, which looks similar to MaterialNode. Of course each purpose is different, but quite big part will overlap each other, I think. Any ideas as to how to factor this out? Handiest ideas would be in the form of a patch ;) 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? It's just deconstructing queries like: WITH t AS (INSERT INTO x ... RETURING *) SELECT * FROM t; to CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *; SELECT * FROM t; While the second statement is not implemented yet, it will be so simpler. So CTAS would get expanded into CTA[row-returning query] ? Interesting. How much work would that part be? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On Fri, Jul 16, 2010 at 12:15 PM, Hitoshi Harada umi.tan...@gmail.com wrote: 2010/7/17 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi: On 7/16/10 6:15 PM +0300, Hitoshi Harada wrote: 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode is exsiting one that work with single tuplestore, it might be sane to modify this so that it accepts tuplestore from Query instead of its child node. I thought about this, but I don't necessarily like the idea of overloading executor nodes. Neither do I have good shape for this solution. Maybe it's not good idea. But my concern is adding DtScanNode, which looks similar to MaterialNode. Of course each purpose is different, but quite big part will overlap each other, I think. 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? It's just deconstructing queries like: WITH t AS (INSERT INTO x ... RETURING *) SELECT * FROM t; to CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *; SELECT * FROM t; Is it acceptable for a wCTE query to manipulate the system catalogs? Couldn't this cause performance issues in some cases? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On Tue, Jul 20, 2010 at 5:13 PM, Merlin Moncure mmonc...@gmail.com wrote: 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? It's just deconstructing queries like: WITH t AS (INSERT INTO x ... RETURING *) SELECT * FROM t; to CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *; SELECT * FROM t; Is it acceptable for a wCTE query to manipulate the system catalogs? Couldn't this cause performance issues in some cases? Yeah, I suspect the performance wouldn't be too hot. I think the idea of writing into a tuplestore and then reading back out from it is the way to go. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
2010/7/13 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi: Hi, I've been working on writeable CTEs during the last couple of months, but right now it looks like I'm going to miss the first commit fest for 9.1. I was trying to make it work by expanding all wCTEs to their own Queries during the rewrite stage (a very crude patch trying to do that for regular CTEs attached), but I don't think that it's a good way of approaching the problem. Consider: Sorry it's not relevant to the topic you post but it seems you're going to add new executor node called DtScanNode and let it hold tuplestore passed by the Query indicated by index number. However, I suppose there are other ways: 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode is exsiting one that work with single tuplestore, it might be sane to modify this so that it accepts tuplestore from Query instead of its child node. 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I'm not familiar with the long discussion on this feature so not sure they are possible, but ISTM they are enough to be discussed (or discussed already?). Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On 7/16/10 6:15 PM +0300, Hitoshi Harada wrote: Sorry it's not relevant to the topic you post but .. Relevant enough :-) .. it seems you're going to add new executor node called DtScanNode and let it hold tuplestore passed by the Query indicated by index number. However, I suppose there are other ways: 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode is exsiting one that work with single tuplestore, it might be sane to modify this so that it accepts tuplestore from Query instead of its child node. I thought about this, but I don't necessarily like the idea of overloading executor nodes. 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? I'm not familiar with the long discussion on this feature so not sure they are possible, but ISTM they are enough to be discussed (or discussed already?). We haven't discussed this part of the design yet.. Now is a good time to do it. Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
2010/7/17 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi: On 7/16/10 6:15 PM +0300, Hitoshi Harada wrote: 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode is exsiting one that work with single tuplestore, it might be sane to modify this so that it accepts tuplestore from Query instead of its child node. I thought about this, but I don't necessarily like the idea of overloading executor nodes. Neither do I have good shape for this solution. Maybe it's not good idea. But my concern is adding DtScanNode, which looks similar to MaterialNode. Of course each purpose is different, but quite big part will overlap each other, I think. 2. Use temp table instead of tuplestore list. Since we agreed we need to execute each plan one by one starting and shutting down executor, it now looks very simple strategy. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? It's just deconstructing queries like: WITH t AS (INSERT INTO x ... RETURING *) SELECT * FROM t; to CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *; SELECT * FROM t; While the second statement is not implemented yet, it will be so simpler. Another concern is tuplestore's memory exhausting. Tuplestore holds tuples in memory as far as the estimated memory usage is within work_mem (for *each* not total of all tuplestores!), but if you create dozens of tuplestore (and it's quite possible in wCTE use cases) we will possibly fail into memory overflow problems. I'm not familiar with the long discussion on this feature so not sure they are possible, but ISTM they are enough to be discussed (or discussed already?). We haven't discussed this part of the design yet.. Now is a good time to do it. Yeah, we should. Anyone has another idea? Or adding DtScanNode for this features is fair enough? Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On 7/16/10 7:15 PM +0300, Hitoshi Harada wrote: 2010/7/17 Marko Tiikkajamarko.tiikk...@cs.helsinki.fi: I thought about this, but I don't necessarily like the idea of overloading executor nodes. Neither do I have good shape for this solution. Maybe it's not good idea. But my concern is adding DtScanNode, which looks similar to MaterialNode. Of course each purpose is different, but quite big part will overlap each other, I think. The way I see it is that reading from a tuplestore is so simple that we shouldn't be trying to merge together nodes just on that basis. It seems to me that we'd have to add CteScan and WorkTableScan nodes there too and at that point it would become complicated. I didn't look at this because I thought using a tuplestore receiver in the portal logic was simple enough. Any thoughts on how this would work? It's just deconstructing queries like: WITH t AS (INSERT INTO x ... RETURING *) SELECT * FROM t; to CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *; SELECT * FROM t; That's an idea. Can we somehow avoid name clashes with user-defined temporary tables? Another concern is tuplestore's memory exhausting. Tuplestore holds tuples in memory as far as the estimated memory usage is within work_mem (for *each* not total of all tuplestores!), but if you create dozens of tuplestore (and it's quite possible in wCTE use cases) we will possibly fail into memory overflow problems. That doesn't seem very different from a big SELECT query, except with wCTEs, you actually *know* how many times the work_mem can be used before you run the query and can adjust work_mem accordingly. That said, I personally could live with a separate GUC for just adjusting the work_mem of wcte tuplestores. Another option would be to unconditionally force the tuplestores to disk, but that sounds painful. Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On 7/12/10 9:07 PM +0300, I wrote: Consider: WITH t AS (SELECT 1), t2 AS (SELECT * FROM t2) VALUES (true); That should of course have been SELECT * FROM t, not t2. Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes: ... So what I'm now thinking of is making the planner plan that as a single Query, and make the planner expand it into multiple PlannedStmts if necessary. This would break the existing planner hooks, but I don't think that's a huge problem. Does anyone see any obvious flaws in this? How will that interact with the existing rewriter? It sounds a lot like a recipe for duplicating functionality ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on writeable CTEs
On 7/12/10 9:34 PM +0300, Tom Lane wrote: Marko Tiikkajamarko.tiikk...@cs.helsinki.fi writes: ... So what I'm now thinking of is making the planner plan that as a single Query, and make the planner expand it into multiple PlannedStmts if necessary. This would break the existing planner hooks, but I don't think that's a huge problem. Does anyone see any obvious flaws in this? How will that interact with the existing rewriter? It sounds a lot like a recipe for duplicating functionality ... I was thinking that the rewriter would look at the top-level CTEs and rewrite all non-SELECT queries into a list of Queries and store that list into the CommonTableExprs. The planner would then use this information and also expand the rewrite products. Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report: getting plpgsql to use the core lexer
Tom Lane t...@sss.pgh.pa.us wrote: One problem that wasn't obvious when I started is that if you are trying to use a reentrant lexer, Bison insists on including its YYSTYPE union in the call signature of the lexer. Of course, YYSTYPE means different things to the core grammar and plpgsql's grammar. I tried to work around that by having an interface layer that would (among other duties) translate as needed. It turned out to be a real PITA, not least because you can't include both definitions into the same C file. The scheme I have has more or less failed --- I think I'd need *two* interface layers to make it work without unmaintainable kluges. It would probably be better to try to adjust the core lexer's API some more so that it does not depend on the core YYSTYPE, but I'm not sure yet how to get Bison to play along without injecting an interface layer (and hence wasted cycles) into the core grammar/lexer interface. Another pretty serious issue is that the current plpgsql lexer treats various sorts of qualified names as single tokens. I had thought this could be worked around in the interface layer by doing more lookahead. You can do that, and it mostly works, but it's mighty tedious. The big problem is that yytext gets out of step --- it will point at the last token the core lexer has processed, and there's no good way to back it up after lookahead. I spent a fair amount of time trying to work around that by eliminating uses of yytext in plpgsql, and mostly succeeded, but there are still some left. (Some of the remaining regression failures are error messages that point at the wrong token because they rely on yytext.) Now, having name lookup happen at the lexical level is pretty bogus anyhow. The long-term solution here is probably to avoid doing lookup in the plpgsql lexer and move it into some sort of callback hook in the main parser, as we've discussed before. I didn't want to get into that right away, but I'm now thinking it has to happen before not after refactoring the lexer code. One issue that has to be surmounted before that can happen is that plpgsql currently throws away all knowledge of syntactic scope after initial processing of a function --- the name stack is no longer available when we want to parse individual SQL commands. We can probably rearrange that design but it's another bit of work I don't have time for right now. All of this sounds pretty familiar to me. As you may recall, our framework includes a SQL parser which parses the subset of standard SQL we feel is portable enough, and generates Java classes to implement the code in lowest common denominator SQL with all procedural code for triggers and stored procedures handled in Java (which runs in our middle tier database service). We use ANTLR, and initially had a three-phase process: lexer, parser, and tree-walkers to generate code. We were doing way too much in the parser phase -- checking for table names, column names, data types, etc. The syntax of SQL forced us to do a lot of scanning forward and remembering where we were (especially to get the FROM clause information so we could process columns in the result list). We were able to get to much cleaner code by rewriting the parser to have a dumb phase to get the overall structure into an AST, and then use a tree-walker phase to do all the lookups and type resolution after we had the rough structure, writing another AST to walk for code generation. Besides making the code cleaner and easier to maintain, it helped us give better error messages pointing more accurately to the source of the problem. I don't know if a similar approach is feasible in flex/bison, but if it is, refactoring for an extra pass might be worth the trouble. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report: getting plpgsql to use the core lexer
Kevin Grittner kevin.gritt...@wicourts.gov writes: ... We were able to get to much cleaner code by rewriting the parser to have a dumb phase to get the overall structure into an AST, and then use a tree-walker phase to do all the lookups and type resolution after we had the rough structure, writing another AST to walk for code generation. Besides making the code cleaner and easier to maintain, it helped us give better error messages pointing more accurately to the source of the problem. I don't know if a similar approach is feasible in flex/bison, but if it is, refactoring for an extra pass might be worth the trouble. That's actually what we have in the core parser. plpgsql is trying to take shortcuts, and this whole project is exactly about weaning it away from that. The bottom line is I tried to tackle the sub-projects in the wrong order... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report: getting plpgsql to use the core lexer
Tom Lane wrote: Another pretty serious issue is that the current plpgsql lexer treats various sorts of qualified names as single tokens. I had thought this could be worked around in the interface layer by doing more lookahead. You can do that, and it mostly works, but it's mighty tedious. The big problem is that yytext gets out of step --- it will point at the last token the core lexer has processed, and there's no good way to back it up after lookahead. I spent a fair amount of time trying to work around that by eliminating uses of yytext in plpgsql, and mostly succeeded, but there are still some left. (Some of the remaining regression failures are error messages that point at the wrong token because they rely on yytext.) Just wondering if there are additional regressions not detected due to pg_regress using the ignore-whitespace option to diff. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report: getting plpgsql to use the core lexer
Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: ... I spent a fair amount of time trying to work around that by eliminating uses of yytext in plpgsql, and mostly succeeded, but there are still some left. (Some of the remaining regression failures are error messages that point at the wrong token because they rely on yytext.) Just wondering if there are additional regressions not detected due to pg_regress using the ignore-whitespace option to diff. Good question, but I doubt it --- those messages all use double quotes around the yytext string, and I believe that eg. foo and foo are different even under --ignore-whitespace. I just finished wiping all that stuff from my work directory, or I'd be able to give you a non-guesswork answer :-( ... but it's not worth regenerating the build for. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status Report on SE-PostgreSQL
The patch set of SE-PostgreSQL and related stuff were updated (r1467). [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1467.patch [2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1467.patch [3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1467.patch [4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1467.patch [5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1467.patch List of updates: - Documentation is updated based on Robert Haas's suggestions. See, http://archives.postgresql.org/message-id/497c6808.2060...@kaigai.gr.jp - Bugfix: SE-PostgreSQL related functions didn't raise an error when pgace_feature option is not selinux. - Add a launcher program to run testcases with various kind of security context. Thanks, -- KaiGai Kohei kai...@kaigai.gr.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status Report on Hot Standby
Robert Treat wrote: On Friday 16 January 2009 19:16:42 Simon Riggs wrote: Bruce asked for 2 more weeks to get patches into shape for commit. Current patch v8e is attached here. Ready for commit? Up to you. My overall opinion is that it's in very good shape. Worth the community including it in this release and spending further time on it. I'm happy to stand by this going forwards. +1 +1 I've been testing several versions of this patch, and overall it looks very good. regards Mark -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status Report on Hot Standby
On Friday 16 January 2009 19:16:42 Simon Riggs wrote: Bruce asked for 2 more weeks to get patches into shape for commit. Current patch v8e is attached here. Ready for commit? Up to you. My overall opinion is that it's in very good shape. Worth the community including it in this release and spending further time on it. I'm happy to stand by this going forwards. +1 The patch could benefit further from input from other various hackers, what couldn't? It's time to put this in a shared repository (of some kind) and make further changes to it in a controlled manner. Yep. I've now got this running on Linux and Solaris and testing so far has looked good. I've also spoken to a couple other people who have built it and run it, and everyone has been pretty happy. It'd certainly be nice to see this get into the main source tree to make it easier for future testing. (For example, one hurdle on Solaris, I had to get a different version of patch to handle Simon's diff... ugh!) -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status Report on Hot Standby
On Tue, 2009-01-20 at 18:08 -0500, Robert Treat wrote: The patch could benefit further from input from other various hackers, what couldn't? It's time to put this in a shared repository (of some kind) and make further changes to it in a controlled manner. Yep. I've now got this running on Linux and Solaris and testing so far has looked good. I've also spoken to a couple other people who have built it and run it, and everyone has been pretty happy. It'd certainly be nice to see this get into the main source tree to make it easier for future testing. (For example, one hurdle on Solaris, I had to get a different version of patch to handle Simon's diff... ugh!) There'll be a new version out soon, hopefully tomorrow, with some fancy new deferred conflict resolution. Timing depends on passage through testing. I think it will then be fermented in GIT for a while. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status report on 8.3 release
On Thu, 2007-11-29 at 11:26 -0500, Bruce Momjian wrote: I expect these cleanups to continue for at least another week or two. Once they slow we will schedule RC1. So are there no plans for an additional beta? Given the recent addition of changes like http://archives.postgresql.org/pgsql-committers/2007-11/msg00552.php http://archives.postgresql.org/pgsql-committers/2007-11/msg00532.php I wonder if another beta before RC1 would be warranted. -Neil ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Status report on 8.3 release
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 29 Nov 2007 14:01:11 -0800 Neil Conway [EMAIL PROTECTED] wrote: On Thu, 2007-11-29 at 11:26 -0500, Bruce Momjian wrote: I expect these cleanups to continue for at least another week or two. Once they slow we will schedule RC1. So are there no plans for an additional beta? Given the recent addition of changes like http://archives.postgresql.org/pgsql-committers/2007-11/msg00552.php http://archives.postgresql.org/pgsql-committers/2007-11/msg00532.php I wonder if another beta before RC1 would be warranted. I think the email is not quite clear :). There is a Beta 4 coming, the problem is packagers aren't really available this week so if it comes it will be next week. At least that is last I heard. Joshua D. Drake -Neil ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster - -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHTzhHATb/zqfZUUQRAiZlAJ0Tj/Kzn0cFnfm7pB9YFvKQm4txpQCfTCo6 s6FF7Ey5GMISxNmwxTlgi7g= =KypU -END PGP SIGNATURE- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Status report on 8.3 release
Neil Conway [EMAIL PROTECTED] writes: So are there no plans for an additional beta? Yes, there are, but not till we do something about http://archives.postgresql.org/pgsql-hackers/2007-11/msg01302.php regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Status report on 8.3 release
Andreas 'ads' Scherbaum [EMAIL PROTECTED] writes: i would also like to test another Beta, if we do something about this problem: http://archives.postgresql.org/pgsql-hackers/2007-11/msg00960.php That's already done, it would be in the next beta. You could check out a copy from CVS HEAD if you want to test it now. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support! ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Status report on 8.3 release
Hello, On Thu, 29 Nov 2007 17:21:09 -0500 Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: So are there no plans for an additional beta? Yes, there are, but not till we do something about http://archives.postgresql.org/pgsql-hackers/2007-11/msg01302.php i would also like to test another Beta, if we do something about this problem: http://archives.postgresql.org/pgsql-hackers/2007-11/msg00960.php Kind regards -- Andreas 'ads' Scherbaum German PostgreSQL User Group ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Status report on 8.3 releaset
Joshua D. Drake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 29 Nov 2007 14:01:11 -0800 Neil Conway [EMAIL PROTECTED] wrote: On Thu, 2007-11-29 at 11:26 -0500, Bruce Momjian wrote: I expect these cleanups to continue for at least another week or two. Once they slow we will schedule RC1. So are there no plans for an additional beta? Given the recent addition of changes like http://archives.postgresql.org/pgsql-committers/2007-11/msg00552.php http://archives.postgresql.org/pgsql-committers/2007-11/msg00532.php I wonder if another beta before RC1 would be warranted. I think the email is not quite clear :). There is a Beta 4 coming, the problem is packagers aren't really available this week so if it comes it will be next week. Yes, there will be another beta before RC1. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Status report on 8.3 release
On Nov 30, 2007 11:10 AM, Andreas 'ads' Scherbaum [EMAIL PROTECTED] wrote: i would also like to test another Beta, if we do something about this problem: http://archives.postgresql.org/pgsql-hackers/2007-11/msg00960.php Hi Andreas, Tom's already committed the quote_literal(anyelement) function. http://archives.postgresql.org/pgsql-committers/2007-11/msg00530.php Cheers BJ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Status report on 8.3 release
On Fri, 30 Nov 2007 11:26:35 +1100 Brendan Jurd wrote: On Nov 30, 2007 11:10 AM, Andreas 'ads' Scherbaum [EMAIL PROTECTED] wrote: i would also like to test another Beta, if we do something about this problem: http://archives.postgresql.org/pgsql-hackers/2007-11/msg00960.php Hi Andreas, Tom's already committed the quote_literal(anyelement) function. http://archives.postgresql.org/pgsql-committers/2007-11/msg00530.php Ups, i've overseen this one. Forget my posting and i will keep testing the next beta ;-) Kind regards -- Andreas 'ads' Scherbaum German PostgreSQL User Group ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
On 7/13/2004 10:23 PM, Christopher Kings-Lynne wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think you guys just need to learn to become comfortable with being hard-nosed about this. Just Do Not Care about people who want ARC right this second. Do you see them calling up Oracle and saying please backport the new stuff from 11i-devel so we can use it now please? Hmmm ... I wonder what you think who those people are who want ARC right this second, and who you guys are on the other hand. To fill you in a little, I am the PostgreSQL CORE team member Jan Wieck, who burned Afilias payroll hours to implement the ARC buffer replacement strategy. The feature has been completed and fully contributed under the BSD license way ahead of any possible release schedule. I have had several requests for backports, but consider the feature too delicate to make such patch available. Don't worry, it's not one of my goals to get this into any release, the reason I am personally touched by this is not because it affects my checking account in any way. What touches me here is the fact that the PostgreSQL Open Source Project under the BSD license seems starting to care a lot more about some press releases and silly news splashes, than to care about real features contributed under the terms and conditions of the BSD license by serious members of the Open Source Community. Which part of the storage manager, that Futjitsu uses so that their customers don't need ARC, the background writer or vacuum at all, do you think will be contributed any time soon? Don't get me wrong here, I don't have a problem with someone making money out of my 8+ years of contributions to this project. But I do have a problem with those efforts getting in my way of contributing. Come again about hard-nosing, please. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
Jan Wieck wrote: On 7/13/2004 10:23 PM, Christopher Kings-Lynne wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think you guys just need to learn to become comfortable with being hard-nosed about this. Just Do Not Care about people who want ARC right this second. Do you see them calling up Oracle and saying please backport the new stuff from 11i-devel so we can use it now please? Hmmm ... I wonder what you think who those people are who want ARC right this second, and who you guys are on the other hand. To fill you in a little, I am the PostgreSQL CORE team member Jan Wieck, who burned Afilias payroll hours to implement the ARC buffer replacement strategy. The feature has been completed and fully contributed under the BSD license way ahead of any possible release schedule. I have had several requests for backports, but consider the feature too delicate to make such patch available. Don't worry, it's not one of my goals to get this into any release, the reason I am personally touched by this is not because it affects my checking account in any way. What touches me here is the fact that the PostgreSQL Open Source Project under the BSD license seems starting to care a lot more about some press releases and silly news splashes, than to care about real features contributed under the terms and conditions of the BSD license by serious members of the Open Source Community. Which part of the storage manager, that Futjitsu uses so that their customers don't need ARC, the background writer or vacuum at all, do you think will be contributed any time soon? Don't get me wrong here, I don't have a problem with someone making money out of my 8+ years of contributions to this project. But I do have a problem with those efforts getting in my way of contributing. What are you talking about? Are you suggesting Fujitsu's features are getting more attendtion than ARC for some political reason? You think nested transactions and tablespaces are just press release features? All those features are being developed by the community under community direction and based on community feedback/needs. How is that different from Afilias funding ARC? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Release planning (was: Re: [HACKERS] Status report)
On Wed, 14 Jul 2004, Bruce Momjian wrote: What are you talking about? Are you suggesting Fujitsu's features are getting more attendtion than ARC for some political reason? You think nested transactions and tablespaces are just press release features? All those features are being developed by the community under community direction and based on community feedback/needs. How is that different from Afilias funding ARC? No, the beef that Jan has, and that I also have, is that we put off the release that was schedualed for June 1st in order to get Nested Xacts into the tree ... hindsight being 20-20, we shouldn't have done that, but should have delayed Nested Xacts for 7.6 ... there *were* enough features in the tree to warrant a release, and features that ppl needed / wanted. Do I believe there were political motivations for postponing the feature freeze? Personally ... no. And I don't believe that the Press Release (and a nice one at that) can really be counted as motivation for the postponement, since the PR was done *after* we decided to push things back a month ... I do believe that there was some pressure from Futjitsu involved, in postponing it, since they'd rather see it in sooner then later ... *but* ... I don't really believe that the pressure is any different then there quite possibly could have been had, say, Jan been almost finished ARC and Affilias wanted to see that sooner rather then later ... we all want to see the feature we've either been working on, or funded, released as soon as possible ... The big problem that I see with how this feature freeze/beta/release has gone down is that we have *alot* of big items that are/were being worked on (ARC, BGWriter, auto_vacuum, PITR, Nested Xacts), and only so much man power at the reviewer stage ... we *should* have frozen it all on June 1st, got the ready features out the door and released, and then concentrated on getting the almost ready, but not quite features into the next release as quickly as possible ... Hindsight is 20-20 ... maybe next time we'll learn from it? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
On 7/14/2004 1:13 PM, Bruce Momjian wrote: What are you talking about? Are you suggesting Fujitsu's features are getting more attendtion than ARC for some political reason? You think nested transactions and tablespaces are just press release features? All those features are being developed by the community under community direction and based on community feedback/needs. How is that different from Afilias funding ARC? I was getting a little upset about the suggestion of being hard-nosed against the people who want ARC now. Yes, I wanted those features out earlier and this was not because of Afilias or dictated by Afilias, but because there is demand for it from the community. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: The big problem that I see with how this feature freeze/beta/release has gone down is that we have *alot* of big items that are/were being worked on (ARC, BGWriter, auto_vacuum, PITR, Nested Xacts), and only so much man power at the reviewer stage ... we *should* have frozen it all on June 1st, got the ready features out the door and released, and then concentrated on getting the almost ready, but not quite features into the next release as quickly as possible ... Hindsight is 20-20 ... maybe next time we'll learn from it? True, 20-20. One thing is that you can't schedule assuming full knowledge of the future, of course. For example, on May 1, I thought by June 1 PITR and NT would be done, and that Win32 and tablespaces wouldn't. What actually happened is that tablespaces made the deadline (with June adjustments), NT is in but needs more work, and Win32 is better off now than I thought it would be. And we don't know if PITR is ready or not because we haven't studied it enough. One problem with pushing 7.5 out June 1 and then coming back to these features is losing momentum. Sure, ideally we could hit them all in 7.6, but realistically it is a lot harder to get things moving once you stop. Look at the PITR patch we had for 7.3 (?) and are only now getting it into the tree. I am not saying we did things right or wrong --- just pointing out the momentum issue. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 2004-07-13 at 23:03, Marc G. Fournier wrote: As a community, I don't think we should be 'supporting' anything older then the last STABLE ... I agree, but that message isn't clearly stated anywhere. The lists are full of people running very old releases - and everybody keeps having to ask what release are you running and the answer is always shocking. Can we set up different lists for the various major versions, so people can see where to go for different types of support? Rather than General, have General7.4 and General7.3...making everybody's first question what is my release? where should I post this question? - 95% of people are newbies, whatever the list/project. That will also make it clear that on-list support is available for older releases, but they really should be thinking about upgrade. And different people can choose whether to hang out at the bleeding edge, or legacy support. We could also have different hints and tips by list then. One of the gems (hint 9, I think) is changing in r7.5, so we must either mis-advise people who use the new release or fail to advise those using an older release. The down-rev lists could contain advice about upgrading... Best Regards, Simon Riggs ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
On Wed, 2004-07-14 at 21:02, Bruce Momjian wrote: Marc G. Fournier wrote: The big problem that I see with how this feature freeze/beta/release has gone down is that we have *alot* of big items that are/were being worked on (ARC, BGWriter, auto_vacuum, PITR, Nested Xacts), and only so much man power at the reviewer stage ... we *should* have frozen it all on June 1st, got the ready features out the door and released, and then concentrated on getting the almost ready, but not quite features into the next release as quickly as possible ... Hindsight is 20-20 ... maybe next time we'll learn from it? True, 20-20. One thing is that you can't schedule assuming full knowledge of the future, of course. For example, on May 1, I thought by June 1 PITR and NT would be done, and that Win32 and tablespaces wouldn't. What actually happened is that tablespaces made the deadline (with June adjustments), NT is in but needs more work, and Win32 is better off now than I thought it would be. And we don't know if PITR is ready or not because we haven't studied it enough. I see a couple of issues: - new releases of PostgreSQL require a full initdb. There is little upgrade support as there is with other products. (Not complaining..) - commercial products release around every 18 months...customers do NOT want this to be any quicker...more disruptive upgrades, plus marketing takes time to organise. I have customers that upgrade regularly every 3+ years (!), and take longer term strategy into consideration. - commercial issues often cause products to be delayedMS is said to be late delivering Yukonmarketing-led companies often delay waiting for the right feature setsaled-led companies never do. Delivering early as Oracle often does mean that the received wisdom is never upgrade to a x.0 version, always wait for the x.1 minor version where all the features actually work as advertised. Deadlines are great, but advertise them ahead of time, then stick to them. Every other project I see has a big page saying how to contribute and then details of deadlines etc.. We can have a major feature deadline, then a minor feature deadline. I particularly liked the idea of 1 July as the major feature deadline, then 14 July as major-feature-tweak deadline. That funnels things better to cater for the manpower available. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Release planning (was: Re: [HACKERS] Status report)
We can have a major feature deadline, then a minor feature deadline. I particularly liked the idea of 1 July as the major feature deadline, then 14 July as major-feature-tweak deadline. That funnels things better to cater for the manpower available. That being said, your PITR patch still hasn't been comitted and it's well past 1st July. If I had to drop something from 7.5 right now to make a release, then it would have to be PITR! Chris ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Release planning (was: Re: [HACKERS] Status report)
On Thu, 15 Jul 2004, Christopher Kings-Lynne wrote: We can have a major feature deadline, then a minor feature deadline. I particularly liked the idea of 1 July as the major feature deadline, then 14 July as major-feature-tweak deadline. That funnels things better to cater for the manpower available. That being said, your PITR patch still hasn't been comitted and it's well past 1st July. If I had to drop something from 7.5 right now to make a release, then it would have to be PITR! As Bruce would put it that would be unfair to Simon, since it isn't his fault that the patch hasn't been reviewed for commit yet ... if, once reviewed, it is found to need work, then it should be put off until 7.6 ... I'm in Josh's camp on this one, in that Nested Xacts should probably be pulled since it still needs work :( Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Release planning (was: Re: [HACKERS] Status report)
Christopher Kings-Lynne wrote: We can have a major feature deadline, then a minor feature deadline. I particularly liked the idea of 1 July as the major feature deadline, then 14 July as major-feature-tweak deadline. That funnels things better to cater for the manpower available. That being said, your PITR patch still hasn't been comitted and it's well past 1st July. If I had to drop something from 7.5 right now to make a release, then it would have to be PITR! Well, the problem is that Simon isn't the reason the patch hasn't been applied. The reason is that we have had other priorities in reviewing patches so it doesn't seem fair to single out that feature for exclusion. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Jan Wieck wrote: On 7/10/2004 11:02 PM, Bruce Momjian wrote: If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. Marc isn't the only one who didn't like waiting for more features going into 7.5 two months ago. I agree that it is too late now to push things just out. But the mistake made wasn't ours. What this repeated discussion about release plans and schedules (and we have it every release) indicates to me is that we might miss something. As you pointed out elsewhere, a 9-12 month development cycle just isn't enough to get those big features done. But I think that stretching the release cycle to two years and holding back all the smaller features as well isn't a good idea either. I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. PITR could be done that way though. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Release planning (was: Re: [HACKERS] Status report)
On 7/10/2004 11:02 PM, Bruce Momjian wrote: If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. Marc isn't the only one who didn't like waiting for more features going into 7.5 two months ago. I agree that it is too late now to push things just out. But the mistake made wasn't ours. What this repeated discussion about release plans and schedules (and we have it every release) indicates to me is that we might miss something. As you pointed out elsewhere, a 9-12 month development cycle just isn't enough to get those big features done. But I think that stretching the release cycle to two years and holding back all the smaller features as well isn't a good idea either. I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Jan --- Marc G. Fournier wrote: On Sat, 10 Jul 2004, Bruce Momjian wrote: My point is that it isn't the patch submitters we are waiting on anymore. It is the backlog of patches that need review/adjustment. Of course, many of the patches I kept need some adjustment to get applied ... ... to me, that indicates that even after review, they will have to go back to the submitter to be adjusted, and sent back, and reviewed, and ... Get in what you feel comfortable adding over the next week, the rest can wait until 7.6 ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Release planning (was: Re: [HACKERS] Status report)
Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. Totally agree. I think the only downside to a longer release cycle is that features developed would take longer to get out to the public. Perhaps we need to start thinking of add-ons to existing releases such as an ARC or vacuum-delay add-on to the 7.4.X release. The patch would potentially have to be recreated for every minor release. I would also like to see some psql message that shows the add-ons added to an official release. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Release planning (was: Re: [HACKERS] Status report)
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. I think the only downside to a longer release cycle is that features developed would take longer to get out to the public. Perhaps we need to start thinking of add-ons to existing releases such as an ARC or vacuum-delay add-on to the 7.4.X release. Mmm ... for people whose pain-point has to do with compatibility testing, adding features in minor releases would turn them into major releases, because they'd still have to wonder whether the new version would break anything. I think our policy of only bug fixes in stable releases is important to maintain, because it makes it easy for people to upgrade within a stable release series. Also, we do not really have the manpower to test multiple concurrently developed versions ... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. I think the only downside to a longer release cycle is that features developed would take longer to get out to the public. Perhaps we need to start thinking of add-ons to existing releases such as an ARC or vacuum-delay add-on to the 7.4.X release. Mmm ... for people whose pain-point has to do with compatibility testing, adding features in minor releases would turn them into major releases, because they'd still have to wonder whether the new version would break anything. I think our policy of only bug fixes in stable releases is important to maintain, because it makes it easy for people to upgrade within a stable release series. Also, we do not really have the manpower to test multiple concurrently developed versions ... When I say add-ons, I am thinking of source code patches that are avaiable as options to the standard subreleases. I agree we don't want to change our current subrelease criteria. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Release planning (was: Re: [HACKERS] Status report)
Note that I'm all for a long (2 year? or even 'indefinite') development cycle for a major release if we continue on with what has been happening more often lately, where stuff is back patched to the last stable release ... there is alot of stuff that doesn't require a reload of the database (or initdb) that could very easily be backpatched ... As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... On Tue, 13 Jul 2004, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. regards, tom lane Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: Note that I'm all for a long (2 year? or even 'indefinite') development cycle for a major release if we continue on with what has been happening more often lately, where stuff is back patched to the last stable release ... there is alot of stuff that doesn't require a reload of the database (or initdb) that could very easily be backpatched ... As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... Of course that last sentence is the downside too --- people don't want to have to retest their setups after a minor release upgrade. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Release planning (was: Re: [HACKERS] Status report)
On Tuesday 13 July 2004 17:12, Marc G. Fournier wrote: ... there is alot of stuff that doesn't require a reload of the database (or initdb) that could very easily be backpatched ... As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... Split the feature out into either a patch or a module and put it in contrib until the next major version. Let contrib hold the smaller, non-initdb-forcing patches and such until the next major version rolls them into the mainline. Or even a patches tree parallel to contrib. Either way, the patch or module or whatever wouldn't be in the mainline unless the user needed it. Or maybe we need to rethink what a major version is. But even minor things can force an initdb, although we're better than we have been about this. But Tom's assertion is true. We have enough trouble getting patches rolled out; adding parallel branches is just begging for trouble, due to our relatively small resource size. Although, we probably have enough developers at this point to make it happen. Bruce I would want to be the patchmeister for the stable branch. Someone else (with Bruce's oversight, or Tom's, or whoever) could do the patchmunging and review for the development tree. But I want a stable hand on patches that go into the stable tree. The BSD's release something like that, with CURRENT, TESTING, and STABLE, right? (I'm not a big BSD user...) The Linux kernel has parallel development, and a gatekeeper for each stable release (2.0.x, 2.2.x, 2.4.x, and now 2.6.x). A 2.0.x kernel release happened not too long ago, in fact. We probably could have enough volunteers to do something like that. Gatekeeping a stable release would not be as complex as developing the new release, but, again, I would want an experienced hand on the last stable release where the temptation of backporting 'features' is great. I think a gatekeeper for 7.2.x, for example, would have very little to do except once in a great while if and when a serious bug is found. At that point, that gatekeeper can make a release (the current process is too tied up in people, IMO, and that includes the RPM mechanism (which I have every right to criticize!)). -- Lamar Owen Director of Information Technology Pisgah Astronomical Research Institute 1 PARI Drive Rosman, NC 28772 (828)862-5554 www.pari.edu ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Release planning (was: Re: [HACKERS] Status report)
On Tuesday 13 July 2004 08:52 am, Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Take my opinion with a grain of salt, as I'm young and don't have much experience with large, super-stable C projects. However, looking at how the Linux community handles it, I think we can borrow a lot of concepts from them. Let's seperate out into two different communities: Stable and Dev. The stable folks only want patches when necessary. They don't want new features - they don't use them. They don't want anything but security or stability patches. They don't want to be forced to upgrade. New major versions are moved into the stable community, but they don't replace old versions. No one ever says upgrade or else! Each major version is kept by interested parties and patched as needed. They never die, but are abandoned. The dev community wants to try out all kinds of things. They aren't running production code, so they can risk losing data. They don't have to be so sensitive about backwards compatibility and reliability and security. In the dev community, there is the main branch with all the accepted features. When people feel it is time for a new major version, they spend all their effort stabilizing that main branch. When it is finally stable, they create a new major version and hand it off to the stable community. People keep their own versions of postgresql in the dev community. These compete with each other. For instance, if Tom and others don't feel like a feature should go into the main dev branch, that doesn't mean that Bruce won't put it in his own version and encourage people to develop against that. When Bruce can prove that it is a useful feature and it works and is stable enough, he will work on getting it in the main dev branch. I know this isn't the way things have been done. I know that one of the arguments against proposals like this is that it seperates out our pool of developers. Yes, it will do that. But it will also open up development to more people and more experimentation. Maybe in the end, we will have a stable and dev community that is larger than the entire community we have now. -- Jonathan Gardner [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Most people don't have the time to investigate release notes, release policy details, etc. When there are bug fix updates, you use them. When there are feature updates, you use them for the next installation. Anything in between, or more variations added to that, just cause confusion. And frankly, for the examples thrown around that use this kind of policy, I can't see those as being very successful. I don't want to use a stable branch of a database system that still changes for random reasons. And I don't want a current branch that takes years to finalize. More frequent major releases, to the point that we can stem it, lead to more people getting more features earlier, which is good. Any of the other proposal just make things worse in my mind. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier [EMAIL PROTECTED] writes: On Tue, 13 Jul 2004, Lamar Owen wrote: But Tom's assertion is true. We have enough trouble getting patches rolled out; adding parallel branches is just begging for trouble, due to our relatively small resource size. Although, we probably have enough developers at this point to make it happen. Except, we already have parallel branches, else we'd never have made a 7.4.x release ... The point though is that we expend only very minimal effort on maintaining the stable branches. We only back-patch bug fixes, and 99% of the time the patch is nearly verbatim the same change as we developed to fix the same problem in HEAD. If the code involved has changed enough that a significantly different fix would be required, most of the time we simply don't fix the stable branch. And we spend very nearly zero effort on QA for the stable branch --- there's certainly no significant push to get people to beta-test minor releases. If we did anything much in the way of back-porting features then the level of investment in this would have to rise greatly. In fact, if the proposal is to let people pick and choose which back-ported things they install, then we are not talking about just one stable version but 2^N stable versions for N options. We couldn't possibly test every combination. The BSD's release something like that, with CURRENT, TESTING, and STABLE, right? (I'm not a big BSD user...) Yeah, but they don't support mix-and-match feature sets. A back release has only one current version. We could certainly do something along that line if we had a few people willing to be gatekeepers. We'd have to work harder at making the release generation process open and documented though. Right now there are plenty of steps that only you, Bruce, or Lamar (respectively) know how to do... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: We have always recommended upgrading to the most recent minor release, at least as a project policy for support. Also, the upgrade is only stop/install/restart so it is quite easy to do, and folks like the fact they don't have to test for new functionality. Right, and this made sense when we were dealing with a release every 4 months or so ... now we're talking about one every year, or two ... I meant if they are on 7.4.X they should be on 7.4.3. I don't suggest they have to upgrade 7.2.X to 7.4.X. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Tom Lane wrote: We could certainly do something along that line if we had a few people willing to be gatekeepers. We'd have to work harder at making the release generation process open and documented though. Right now there are plenty of steps that only you, Bruce, or Lamar (respectively) know how to do... I think we could do it if we restricted it to *just* one release back ... but I do agree with Peter about ppls motivations for upgrading (bug fixes vs new features) ... we'd have to have a 'two branch stable', one would be only bug fixes, the other would be bug fixes+feature backpatch ... would could get interesting trying to figure out release naming conventions :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
On Wed, 14 Jul 2004, Peter Eisentraut wrote: Marc G. Fournier wrote: Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Most people don't have the time to investigate release notes, release policy details, etc. When there are bug fix updates, you use them. When there are feature updates, you use them for the next installation. Anything in between, or more variations added to that, just cause confusion. And frankly, for the examples thrown around that use this kind of policy, I can't see those as being very successful. I don't want to use a stable branch of a database system that still changes for random reasons. And I don't want a current branch that takes years to finalize. More frequent major releases, to the point that we can stem it, lead to more people getting more features earlier, which is good. Any of the other proposal just make things worse in my mind. 'k, note that I'm not arguing for more (or less) frequent releases ... but, do you have any thoughts on how we can effectively continue with the 'frequent releases' while bringing in the large features like Nested Xacts? We could start looking at 'development branches' for stuff like this, that could be merged up when ready for prime time, but would at least be available in CVS for those wishing to play with them ... ? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Bruce Momjian wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think this would be a bad direction to take ... I can just see us flooded with a bunch of 'the patch didn't apply to my source tree' questions :( It might work, I just have my doubts ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Lamar Owen wrote: But Tom's assertion is true. We have enough trouble getting patches rolled out; adding parallel branches is just begging for trouble, due to our relatively small resource size. Although, we probably have enough developers at this point to make it happen. Except, we already have parallel branches, else we'd never have made a 7.4.x release ... Bruce I would want to be the patchmeister for the stable branch. Someone else (with Bruce's oversight, or Tom's, or whoever) could do the patchmunging and review for the development tree. But I want a stable hand on patches that go into the stable tree. The BSD's release something like that, with CURRENT, TESTING, and STABLE, right? (I'm not a big BSD user...) We have a CURRENT branch where all the 'innovations' are done (SMP re-writes, etc) ... and a STABLE which is a release with stuff patched down from CURRENT *if* applicable ... periodically, a RELEASE is made along either branch, and, some day, what is currently CURRENT will be re-tag'd as -STABLE, and then CURRENT will become a new branch ... So, for instance, the way we number: 7.4.x would be -STABLE HEAD would be -CURRENT once 7.5 is released, it would surplant 7.4 as -STABLE, previous versions would only ever see 'security related patches' and work towards 7.6 would be -CURRENT ... Now, if we went with a 'long term dev cycle', then we might look at 7.x as being -STABLE, while work towards 8.x would be considered -CURRENT ... As a community, I don't think we should be 'supporting' anything older then the last STABLE ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think this would be a bad direction to take ... I can just see us flooded with a bunch of 'the patch didn't apply to my source tree' questions :( It might work, I just have my doubts ... I have my doubts too, of course. :-) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Bruce Momjian wrote: We have always recommended upgrading to the most recent minor release, at least as a project policy for support. Also, the upgrade is only stop/install/restart so it is quite easy to do, and folks like the fact they don't have to test for new functionality. Right, and this made sense when we were dealing with a release every 4 months or so ... now we're talking about one every year, or two ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 2004-07-13 at 13:03 -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. regards, tom lane It sounds like your only taking the point of view of people upgrading previous installations. What about new installations? I'm sure there are hundreds, if not thousands of new installations happening every week. These people are going to grab the latest stable version without a doubt. I think releasing more often would result in features getting tested much more. Then the big installations can see that a major feature has been in two stable releases (even if the time period was only a year) and feel much more comfortable in upgrading. Why would they have to upgrade more often then necessary anyways? Assuming security exploits are back ported of course. -- Mike Benoit [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... Of course that last sentence is the downside too --- people don't want to have to retest their setups after a minor release upgrade. Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Hell, when we have a client that comes to us with a problem, we don't recommend upgrading unless we find something in the RELEASE NOTES to indicate something has been fixed related to their problem ... it isn't a automatic well, upgrade to the latest stable first, and then we can help you ... God, we still have clients using 7.2 servers, cause they've had no reason yet to upgrade to the latest ... it works, why upgrade? is generally the opinion ... We have always recommended upgrading to the most recent minor release, at least as a project policy for support. Also, the upgrade is only stop/install/restart so it is quite easy to do, and folks like the fact they don't have to test for new functionality. One idea would be for us to release 7.5 and 7.5.0.1, and allow 7.5.1 to have minor new features. That way, 7.5.0.[1-9] are no new features, and 7.5.[1-9] are minor improvements against 7.5. Of course this does require more project management, and we already don't have enough manpower. I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Bruce Momjian wrote: The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... Of course that last sentence is the downside too --- people don't want to have to retest their setups after a minor release upgrade. Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Hell, when we have a client that comes to us with a problem, we don't recommend upgrading unless we find something in the RELEASE NOTES to indicate something has been fixed related to their problem ... it isn't a automatic well, upgrade to the latest stable first, and then we can help you ... God, we still have clients using 7.2 servers, cause they've had no reason yet to upgrade to the latest ... it works, why upgrade? is generally the opinion ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Release planning (was: Re: [HACKERS] Status report)
However, looking at how the Linux community handles it, I think we can borrow a lot of concepts from them. I was thinking quite the opposite. The Linux community doesn't exactly have a great track-record for their stable releases. The thoughts behind the process might be good, but do we have examples where it has worked out well? The 2.4 series seems to have been particularly bad for new major issues in their stable releases. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 2004-07-13 at 20:49, Marc G. Fournier wrote: On Tue, 13 Jul 2004, Tom Lane wrote: We could certainly do something along that line if we had a few people willing to be gatekeepers. We'd have to work harder at making the release generation process open and documented though. Right now there are plenty of steps that only you, Bruce, or Lamar (respectively) know how to do... I think we could do it if we restricted it to *just* one release back ... but I do agree with Peter about ppls motivations for upgrading (bug fixes vs new features) ... we'd have to have a 'two branch stable', one would be only bug fixes, the other would be bug fixes+feature backpatch ... would could get interesting trying to figure out release naming conventions :) FreeBSD does do this. They tag the bug-fix only branches as being SECURITY releases, since typically they only contain fixes for security or reliability related issues. The 5.2.1 release which followed 5.2.0 was an example of one of the few releases the Security team has done. They usually just allow users to use CVSUP to follow the branch. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
God, we still have clients using 7.2 servers, cause they've had no reason yet to upgrade to the latest ... it works, why upgrade? is generally the opinion ... Because there's shocking failure-to-restart bugs in 7.2...and if you upgrade to 7.4 you're forced to get new features as well. Chris ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think you guys just need to learn to become comfortable with being hard-nosed about this. Just Do Not Care about people who want ARC right this second. Do you see them calling up Oracle and saying please backport the new stuff from 11i-devel so we can use it now please? You learn after a long while in the software industry that if left to themselves the users will make _endless_ demands and they think that they NEED everything NOW. Of course in reality, they don't NEED it NOW - they can wait. And even if they did get it now, then it's buggy and then they complain even louder. You cannot win. The arguments about things getting more testing in production is nonsense - we are a major database server - you cannot have users doing the testing for us They have signed on to the PostgreSQL project on our terms of development - if they want to sponsor someone to backport something for their own situation, they can do so. Otherwise, they perhaps should have not been using PostgreSQL for their particular app in the first place! It's not worth thinking along the lines of oh if only we could get this feature out RIGHT NOW, we'd get more users and more people uprading and convertign and better reviews and stuff - but that's a distraction. There will ALWAYS be Just One More Feature that we need to be really great. Just don't worry about it. Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: snip As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... snip Hmmm... so we do things slightly differently than previously... This upcoming version could be PG version 8.0, We continue with bugfixes on 7.4.x, That still leaves 7.5.x, 7.6.x (etc if needed), for releasing new versions of PG without the big features. Kind of like an in-between thing, whilst waiting for the major features in the major releases? That would mean we'd have: Version 8.0 as our next main release, Version 9.0 being the version after that with the next big features in it. Version 8.x being version 8 plus smaller features, prior to 9.0 Version 8.x.x being version 8.x plus bug fixes. Sounds like it could get hairy if we're not careful, but I reckon the PG Community is mature enough to make the right calls where needed. :) Regards and best wishes, Justin Clift ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Status report
Justin Clift wrote: Bruce Momjian wrote: If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. Um, let's take the time to get the features in, otherwise we'll be waiting another year (roughly) to get PITR and others out to end users. We can't affort not having PITR or NT in 7.5; we announced it already everywhere. But it's not a real problem if we need some more time to release, we always tell we don't release according to a release plan, but if things are mature. Regards, Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Status report
Bruce Momjian wrote: If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. Um, let's take the time to get the features in, otherwise we'll be waiting another year (roughly) to get PITR and others out to end users. :) Regards and best wishes, Justin Clift ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Status report
I am still going through my mailbox, trying to address all the open patches so we can move toward beta. Of course, many of the patches I kept need some adjustment to get applied (e.g. configuration file location) so it is slow going. However, we still have PITR unapplied, autovacuum unapplied, and nested transactions don't have savepoint support, so we are still a while away from beta. I think we should start thinking of beta as August 1 rather than mid-July. Plus a bunch of other minor patches :) Chris ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Status report
On Sat, 10 Jul 2004, Bruce Momjian wrote: I am still going through my mailbox, trying to address all the open patches so we can move toward beta. Of course, many of the patches I kept need some adjustment to get applied (e.g. configuration file location) so it is slow going. However, we still have PITR unapplied, autovacuum unapplied, and nested transactions don't have savepoint support, so we are still a while away from beta. I think we should start thinking of beta as August 1 rather than mid-July. And so we fall into the hole we dig for ourselves with each release, where the beta period takes almost as long as the development period itself ... originally, this was all supposed to be done for June 1st, but we pushed it back to July 1st since ppl only needed another month ... then, July 1st, we did the feature freeze, but because so much was left to be applied, we decided to leave bundling first beta until mid-July ... If six extra weeks hasn't been enough, I have little faith that adding 'yet another 2 weeks' is going to be enough either ... leave them for 7.6 then ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Status report
My point is that it isn't the patch submitters we are waiting on anymore. It is the backlog of patches that need review/adjustment. --- Marc G. Fournier wrote: On Sat, 10 Jul 2004, Bruce Momjian wrote: I am still going through my mailbox, trying to address all the open patches so we can move toward beta. Of course, many of the patches I kept need some adjustment to get applied (e.g. configuration file location) so it is slow going. However, we still have PITR unapplied, autovacuum unapplied, and nested transactions don't have savepoint support, so we are still a while away from beta. I think we should start thinking of beta as August 1 rather than mid-July. And so we fall into the hole we dig for ourselves with each release, where the beta period takes almost as long as the development period itself ... originally, this was all supposed to be done for June 1st, but we pushed it back to July 1st since ppl only needed another month ... then, July 1st, we did the feature freeze, but because so much was left to be applied, we decided to leave bundling first beta until mid-July ... If six extra weeks hasn't been enough, I have little faith that adding 'yet another 2 weeks' is going to be enough either ... leave them for 7.6 then ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Status report
On Sat, 10 Jul 2004, Bruce Momjian wrote: My point is that it isn't the patch submitters we are waiting on anymore. It is the backlog of patches that need review/adjustment. Of course, many of the patches I kept need some adjustment to get applied ... ... to me, that indicates that even after review, they will have to go back to the submitter to be adjusted, and sent back, and reviewed, and ... Get in what you feel comfortable adding over the next week, the rest can wait until 7.6 ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Status report
If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. --- Marc G. Fournier wrote: On Sat, 10 Jul 2004, Bruce Momjian wrote: My point is that it isn't the patch submitters we are waiting on anymore. It is the backlog of patches that need review/adjustment. Of course, many of the patches I kept need some adjustment to get applied ... ... to me, that indicates that even after review, they will have to go back to the submitter to be adjusted, and sent back, and reviewed, and ... Get in what you feel comfortable adding over the next week, the rest can wait until 7.6 ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Status report: regex replacement
Tatsuo Ishii writes: UTF-8 seems to be the most popular, but even XML standard requires all compliant implementations to deal with at least both UTF-8 and UTF-16. I don't think PostgreSQL is going to natively support UTF-16. At FOSDEM it was claimed that Windows natively uses UCS-2, and there are also continuing rumours that the Java Unicode encoding is not quite UTF-8, so there is going to be a certain pressure to support other Unicode encodings besides UTF-8. As for the names, the SQL standard defines most of those. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Status report: regex replacement
Tom Lane writes: code is concerned: the regex library actually offers three regex flavors, advanced, extended, and basic, where extended matches what we had before (extended and basic correspond to different levels of the POSIX 1003.2 standard). We just need a way to expose that knob to the user. I am thinking about inventing yet another GUC parameter, say Perhaps it should be exposed through different operators. If someone uses packages (especially functions) provided externally, they might have a hard time coordinating what flavor is required by which part of what he is using. By analogy, imagine there was an environment variable that switched all grep's to egrep's. That would be a complete mess. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Status report: regex replacement
Peter Eisentraut [EMAIL PROTECTED] writes:
Tom Lane writes:
code is concerned: the regex library actually offers three regex
flavors, advanced, extended, and basic, where extended matches
what we had before (extended and basic correspond to different
levels of the POSIX 1003.2 standard). We just need a way to expose
that knob to the user. I am thinking about inventing yet another GUC
parameter, say
Perhaps it should be exposed through different operators. If someone uses
packages (especially functions) provided externally, they might have a
hard time coordinating what flavor is required by which part of what he is
using.
But one could argue the contrary, too: if you've got an
externally-provided package there may be no convenient way to get it to
use, say, ~!#@ in place of ~. GUC variables can come in awfully handy
in scenarios like that.
Also, if one *can* alter the SQL context in which a regexp is used, there
is a solution already provided by Spencer's regex metasyntax hack --- see
http://developer.postgresql.org/docs/postgres/functions-matching.html#POSIX-METASYNTAX
That is, one could write something like
foo ~ ('(?b)' || basic_regex_expression)
to force basic_regex_expression to be taken as a BRE and not the
extended syntax. This is a tad uglier than changing the operator name,
perhaps, but it has some advantages too --- for one, the option could be
plugged into the string further upstream than where the SQL syntax is
determined.
Basically I think the flavor-as-GUC-variable approach is orthogonal to
inventing some new operator names. We could do the latter too, but
I don't really see a need for it given the metasyntax feature.
regards, tom lane
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Status report: regex replacement
Tatsuo Ishii kirjutas R, 07.02.2003 kell 04:03: UTF-8 seems to be the most popular, but even XML standard requires all compliant implementations to deal with at least both UTF-8 and UTF-16. I don't think PostgreSQL is going to natively support UTF-16. By natively, do you mean as backend storage format or as supported client encoding ? -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Status report: regex replacement
I have just committed the latest version of Henry Spencer's regex package (lifted from Tcl 8.4.1) into CVS HEAD. This code is natively able to handle wide characters efficiently, and so it avoids the multibyte performance problems recently exhibited by Wade Klaver. I have not done extensive performance testing, but the new code seems at least as fast as the old, and much faster in some cases. I have tested the new regex with src/test/mb and it all passed. So the new code looks safe at least for EUC_CN, EUC_JP, EUC_KR, EUC_TW, MULE_INTERNAL, UNICODE, though the test does not include all possible regex patterns. -- Tatsuo Ishii ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Status report: regex replacement
On Fri, 7 Feb 2003 00:49, Hannu Krosing wrote: Tatsuo Ishii kirjutas N, 06.02.2003 kell 17:05: Perhaps we should not call the encoding UNICODE but UTF8 (which it really is). UNICODE is a character set which has half a dozen official encodings and calling one of them UNICODE does not make things very clear. Right. Also we perhaps should call LATIN1 or ISO-8859-1 more precisely way since ISO-8859-1 can be encoded in either 7 bit or 8 bit(we use this). I don't know what it is called though. I don't think that calling 8-bit ISO-8859-1 ISO-8859-1 can confuse anybody, but UCS-2 (ISO-10646-1), UTF-8 and UTF-16 are all widely used. UTF-8 seems to be the most popular, but even XML standard requires all compliant implementations to deal with at least both UTF-8 and UTF-16. Strong agreement from me, for whatever value you wish to place on my opinion. UTF-8 is a preferable name to UNICODE. The case for distinguishing 7-bit from 8-bit latin1 seems much weaker. Tim -- --- Tim Allen [EMAIL PROTECTED] Proximity Pty Ltd http://www.proximity.com.au/ http://www4.tpg.com.au/users/rita_tim/ ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Status report: regex replacement
On Thu, 2003-02-06 at 13:25, Tatsuo Ishii wrote: I have just committed the latest version of Henry Spencer's regex package (lifted from Tcl 8.4.1) into CVS HEAD. This code is natively able to handle wide characters efficiently, and so it avoids the multibyte performance problems recently exhibited by Wade Klaver. I have not done extensive performance testing, but the new code seems at least as fast as the old, and much faster in some cases. I have tested the new regex with src/test/mb and it all passed. So the new code looks safe at least for EUC_CN, EUC_JP, EUC_KR, EUC_TW, MULE_INTERNAL, UNICODE, though the test does not include all possible regex patterns. Perhaps we should not call the encoding UNICODE but UTF8 (which it really is). UNICODE is a character set which has half a dozen official encodings and calling one of them UNICODE does not make things very clear. -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Status report: regex replacement
Perhaps we should not call the encoding UNICODE but UTF8 (which it really is). UNICODE is a character set which has half a dozen official encodings and calling one of them UNICODE does not make things very clear. Right. Also we perhaps should call LATIN1 or ISO-8859-1 more precisely way since ISO-8859-1 can be encoded in either 7 bit or 8 bit(we use this). I don't know what it is called though. -- Tatsuo Ishii ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Status report: regex replacement
Tatsuo Ishii kirjutas N, 06.02.2003 kell 17:05: Perhaps we should not call the encoding UNICODE but UTF8 (which it really is). UNICODE is a character set which has half a dozen official encodings and calling one of them UNICODE does not make things very clear. Right. Also we perhaps should call LATIN1 or ISO-8859-1 more precisely way since ISO-8859-1 can be encoded in either 7 bit or 8 bit(we use this). I don't know what it is called though. I don't think that calling 8-bit ISO-8859-1 ISO-8859-1 can confuse anybody, but UCS-2 (ISO-10646-1), UTF-8 and UTF-16 are all widely used. UTF-8 seems to be the most popular, but even XML standard requires all compliant implementations to deal with at least both UTF-8 and UTF-16. -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Status report: regex replacement
Right. Also we perhaps should call LATIN1 or ISO-8859-1 more precisely way since ISO-8859-1 can be encoded in either 7 bit or 8 bit(we use this). I don't know what it is called though. I don't think that calling 8-bit ISO-8859-1 ISO-8859-1 can confuse anybody, but UCS-2 (ISO-10646-1), UTF-8 and UTF-16 are all widely used. I just pointed out that ISO-8859-1 is *not* an encoding, but a character set. UTF-8 seems to be the most popular, but even XML standard requires all compliant implementations to deal with at least both UTF-8 and UTF-16. I don't think PostgreSQL is going to natively support UTF-16. -- Tatsuo Ishii ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Status report: regex replacement
On Wed, 5 Feb 2003, Tom Lane wrote: 1. There are a couple of minor incompatibilities between the advanced regex syntax implemented by this package and the syntax handled by our old code; in particular, backslash is now a special character within bracket expressions. It seems to me that we'd better offer a switch to allow backwards compatibility. This is easily done as far as the code is concerned: the regex library actually offers three regex flavors, advanced, extended, and basic, where extended matches what we had before (extended and basic correspond to different levels of the POSIX 1003.2 standard). We just need a way to expose that knob to the user. I am thinking about inventing yet another GUC parameter, say set regex_flavor = advanced set regex_flavor = extended set regex_flavor = basic [snip] Any suggestions about the name of the parameter? Actually I think 'regex_flavor' sounds fine. Jon ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Status report: regex replacement
set regex_flavor = advanced set regex_flavor = extended set regex_flavor = basic [snip] Any suggestions about the name of the parameter? Actually I think 'regex_flavor' sounds fine. Not more Americanisms in our config files!! :P Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Status report: regex replacement
Christopher Kings-Lynne kirjutas N, 06.02.2003 kell 03:56: set regex_flavor = advanced set regex_flavor = extended set regex_flavor = basic [snip] Any suggestions about the name of the parameter? Actually I think 'regex_flavor' sounds fine. Not more Americanisms in our config files!! :P Maybe support both, like for ANALYZE/ANALYSE ? While at it, could we make another variant - ANALÜÜSI - which would be native for me ;) -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
