Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Thu, Feb 26, 2009 at 7:09 AM, Joshua D. Drake j...@commandprompt.com wrote: I think his reply states that. The long and short is, what Tom was concerned about is true and Heikki has confirmed it. This patch as nice as it would be to have, isn't ready for prime time. It is time to push this patch to 8.5, close up the rest of whatever is left with other patches and move to Beta. The decision on whether it is ready lies with Heikki, and whilst he is prepared to work on it, and there are other patches still in the queue being worked on by other committers there is no reason to bump it yet (unless Heikki wants to look at the other patches). I agree that this patch alone should not delay the release further though. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Thu, Feb 26, 2009 at 3:47 AM, Dave Page dp...@pgadmin.org wrote: I think his reply states that. The long and short is, what Tom was concerned about is true and Heikki has confirmed it. This patch as nice as it would be to have, isn't ready for prime time. It is time to push this patch to 8.5, close up the rest of whatever is left with other patches and move to Beta. The decision on whether it is ready lies with Heikki, and whilst he is prepared to work on it, and there are other patches still in the queue being worked on by other committers there is no reason to bump it yet (unless Heikki wants to look at the other patches). Well, this is a key point. If Heikki thinks that the patch isn't going to be ready for 8.4 (and it sounds like that's the case), then it would be more beneficial to the project for him to lay it aside for a few weeks and help clean out the rest of the patch queue rather than continuing to work on that patch and leaving the other patches to the other committers. Our rate of progress on cleaning out the November CommitFest seems to be asymptotically approaching zero (9 patches committed in December, 6 in January, 4 so far in February...) so benching one of the main committers for a feature that won't be ready anyway is not a good trade-off. Of course, if Heikki can't or isn't interested in working on any of the remaining patches, then he might as well keep plugging away at Hot Standby rather than leaving it for another day. It can live in git until 8.4 is released and then get merged after the tree is branched, and anyone else who wants to do further development in that area can get started knowing what the final shape of that code will be. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Robert Haas wrote: On Thu, Feb 26, 2009 at 3:47 AM, Dave Page dp...@pgadmin.org wrote: I think his reply states that. The long and short is, what Tom was concerned about is true and Heikki has confirmed it. This patch as nice as it would be to have, isn't ready for prime time. It is time to push this patch to 8.5, close up the rest of whatever is left with other patches and move to Beta. The decision on whether it is ready lies with Heikki, and whilst he is prepared to work on it, and there are other patches still in the queue being worked on by other committers there is no reason to bump it yet (unless Heikki wants to look at the other patches). Well, this is a key point. If Heikki thinks that the patch isn't going to be ready for 8.4 (and it sounds like that's the case), then it would be more beneficial to the project for him to lay it aside for a few weeks and help clean out the rest of the patch queue rather than continuing to work on that patch and leaving the other patches to the other committers. Our rate of progress on cleaning out the November CommitFest seems to be asymptotically approaching zero (9 patches committed in December, 6 in January, 4 so far in February...) so benching one of the main committers for a feature that won't be ready anyway is not a good trade-off. Agreed, we could use Heikki's help in closing the other outstanding patches, and if hot standby isn't going to make 8.4, which I agree with, it would be good for him to put hot standby aside and help on other patches. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 11:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Tue, Jan 27, 2009 at 3:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: I already pointed out some pretty serious problems with the updatable views patch. Are you claiming they are trivial to fix? Not at all. I think the deferral of that particular patch is the correct thing to do because there are confirmed, real problems with it that are not realistic to fix in an appropriate timeframe for the release. The primary case that I'm objecting to is HS which you've been saying will take 10 - 12 months to complete having by your own admission not looked at the code or followed the discussion particularly closely. Well, perhaps I'm being pessimistic, or perhaps you're being optimistic. What is undeniable fact is that HS will not be committable this week, which will make it three months since feature freeze. As for when it *will* be committable --- Heikki is saying two weeks if no new problems crop up, but given the rate at which new problems have been found so far, what are the odds of that? We've seen this movie before. Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. I think Tom wins this bet, since it's now been four weeks. In fact, he was being optimistic: the loose ends aren't cleaned up either (based on a review of the wiki, we're about half way there: six patches have been committed in the intervening time and seven remain in the queue). ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Wed, 2009-02-25 at 22:56 -0500, Robert Haas wrote: Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. I think Tom wins this bet, since it's now been four weeks. In fact, he was being optimistic: the loose ends aren't cleaned up either (based on a review of the wiki, we're about half way there: six patches have been committed in the intervening time and seven remain in the queue). I agree. It is time to move on. Joshua D. Drake ...Robert -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Joshua D. Drake wrote: On Wed, 2009-02-25 at 22:56 -0500, Robert Haas wrote: Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. I think Tom wins this bet, since it's now been four weeks. In fact, he was being optimistic: the loose ends aren't cleaned up either (based on a review of the wiki, we're about half way there: six patches have been committed in the intervening time and seven remain in the queue). I agree. It is time to move on. Huh? What features are you saying about to be moved on? Is it the updatable-view feature? or rest of all pending ones? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Thu, 2009-02-26 at 14:17 +0900, KaiGai Kohei wrote: Joshua D. Drake wrote: On Wed, 2009-02-25 at 22:56 -0500, Robert Haas wrote: Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. I think Tom wins this bet, since it's now been four weeks. In fact, he was being optimistic: the loose ends aren't cleaned up either (based on a review of the wiki, we're about half way there: six patches have been committed in the intervening time and seven remain in the queue). I agree. It is time to move on. Huh? What features are you saying about to be moved on? Is it the updatable-view feature? or rest of all pending ones? Tom originally stated (as I recall, no flames please) that we would wait for 2 weeks for the hot standby stuff. It has now been four. That is what I and I believe Robert Haas are talking about. Sincerely, Joshua D. Drake Thanks, -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Joshua D. Drake wrote: On Thu, 2009-02-26 at 14:17 +0900, KaiGai Kohei wrote: Joshua D. Drake wrote: On Wed, 2009-02-25 at 22:56 -0500, Robert Haas wrote: Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. I think Tom wins this bet, since it's now been four weeks. In fact, he was being optimistic: the loose ends aren't cleaned up either (based on a review of the wiki, we're about half way there: six patches have been committed in the intervening time and seven remain in the queue). I agree. It is time to move on. Huh? What features are you saying about to be moved on? Is it the updatable-view feature? or rest of all pending ones? Tom originally stated (as I recall, no flames please) that we would wait for 2 weeks for the hot standby stuff. It has now been four. That is what I and I believe Robert Haas are talking about. Thanks, it helps me clear. P.S, I would like folks not to forget SE-PostgreSQL. -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Thu, 2009-02-26 at 14:43 +0900, KaiGai Kohei wrote: Joshua D. Drake wrote: Tom originally stated (as I recall, no flames please) that we would wait for 2 weeks for the hot standby stuff. It has now been four. That is what I and I believe Robert Haas are talking about. Thanks, it helps me clear. P.S, I would like folks not to forget SE-PostgreSQL. I assure you that I have not. Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Robert Haas wrote: On Tue, Jan 27, 2009 at 11:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: As for when it *will* be committable --- Heikki is saying two weeks if no new problems crop up, but given the rate at which new problems have been found so far, what are the odds of that? We've seen this movie before. Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. I think Tom wins this bet, since it's now been four weeks. In fact, he was being optimistic: the loose ends aren't cleaned up either (based on a review of the wiki, we're about half way there: six patches have been committed in the intervening time and seven remain in the queue). Agreed. Simon has finished the pending items he had four weeks ago, but the code clearly isn't ready for commit yet as new issues are cropping up. And I think the way subtransactions are handled, which has been a difficult part of the patch all along, still needs more thinking. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Heikki, Agreed. Simon has finished the pending items he had four weeks ago, but the code clearly isn't ready for commit yet as new issues are cropping up. And I think the way subtransactions are handled, which has been a difficult part of the patch all along, still needs more thinking. Are there issues other than subtransactions? --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Wed, 2009-02-25 at 22:11 -0800, Josh Berkus wrote: Heikki, Agreed. Simon has finished the pending items he had four weeks ago, but the code clearly isn't ready for commit yet as new issues are cropping up. And I think the way subtransactions are handled, which has been a difficult part of the patch all along, still needs more thinking. Are there issues other than subtransactions? I think his reply states that. The long and short is, what Tom was concerned about is true and Heikki has confirmed it. This patch as nice as it would be to have, isn't ready for prime time. It is time to push this patch to 8.5, close up the rest of whatever is left with other patches and move to Beta. Sincerely, Joshua D. Drake --Josh -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Josh Berkus wrote: Agreed. Simon has finished the pending items he had four weeks ago, but the code clearly isn't ready for commit yet as new issues are cropping up. And I think the way subtransactions are handled, which has been a difficult part of the patch all along, still needs more thinking. Are there issues other than subtransactions? Subtransactions, and transaction tracking in general. I haven't looked at the other parts in detail yet. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Sunday 25 January 2009 19:06:50 Tom Lane wrote: Particularly with regard to hot standby, which by any sane reading was not close to being committable on 1 November (a fortiori from the fact that it's *still* not committable despite large amounts of later work). I'm also feeling that we are not in a position to commit SE-Postgres in a timely fashion; which is not KaiGai-san's fault, rather that of the community which has taken nearly zero interest in that patch. If we want to ensure that 8.5 development opens soon, what we have to do is reject those two patches, revert updatable views, and finish up the other stuff (which is all small and could likely be dealt with in a week or two). Updatable views is reverted. I agree that we should reject the rest and prepare a release. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 2:01 PM, Peter Eisentraut pete...@gmx.net wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. That will send a fine message to those companies that have sponsored development work - that we will arbitrarily reject large patches that have been worked on following the procedures that we require. We must at least have the solid belief (of a committer that that has done a proper review) that a patch cannot be polished in an appropriate timeframe, or another justifiable reason for rejecting rather than vague handwaving, guesswork and estimates based on email traffic. If we do not, we will rapidly find that no company wants to sponsor features for PostgreSQL in the future for fear that their money will be wasted even if they jump through all the right hoops. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Dave Page wrote: On Mon, Jan 26, 2009 at 11:35 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I'm sure it depends on the user. Users that are more interested in the features we already have in the bag like window functions and WITH-clause, will obviously prefer to release earlier without hot standby. And users that want hot standby (or SE-postgresql) will prefer to delay the release and have those features included. At LinuxLive (UK) the overwhelming majority of people I spoke to over three days wanted hot standby and replication (preferably multi-master, but thats another story). Window functions, recursive queries, SE PostgreSQL, updatable views and other new features were barely mentioned. I'd say the FSM rework is the largest feature in 8.4 that most of my customers would have immediate use for. With visibility map not far behind. It's not a sexy feature, it's removing a piece of annoyance. So it's not something people would think of when you say what feature are you looking for in the next version. But it will help pretty much all users, and that's certainly more than hot standby for example. And the longer we push that back for some other feature, the more these users suffer. That said, I've certainly got a fair number of places where hot standby would be very popular. More than most others on your list above. And I'm not making a comment as to how ready hot standby is - I haven't kept up enough to comment on that. As I've pointed out before, we're not a commercial company working for our shareholders, we're a FOSS project working for our end users. If we can include an important and popular feature like this at the expense of a few weeks extra wait for the release, it seems to me that we'll be serving our users far better overall than making a fair percentage of them wait another 12 months for work that is more or less complete. Just playing the devils advocate here, but you can turn that argument around easily. We're not a commercial company who need to release a feature just because marketing said it'd be there... //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Dave Page wrote: On Tue, Jan 27, 2009 at 2:01 PM, Peter Eisentraut pete...@gmx.net wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. That will send a fine message to those companies that have sponsored development work - that we will arbitrarily reject large patches that have been worked on following the procedures that we require. To some extent that seems more an issue of linguistics and tone. If Peter had written we should defer the rest and try to help resolve specific issues identified in the reviews during commitfest 2009-First, sponsors might be happy rather than upset. I think one can make a strong argument that the features should be moved to the next commit-fest, just so the other patches in that commit fest ( http://wiki.postgresql.org/wiki/CommitFest_2009-First ) don't bit-rot too badly. Whether the community wants to release an 8.4 between commitfest 2008-11 and 2009-First seems to me a largely orthogonal question that would be based more on what demand there is for an 8.4 release and how distracting it would be to do a release between commitfests. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Dave Page dp...@pgadmin.org writes: On Tue, Jan 27, 2009 at 2:01 PM, Peter Eisentraut pete...@gmx.net wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. That will send a fine message to those companies that have sponsored development work - that we will arbitrarily reject large patches that have been worked on following the procedures that we require. defer to 8.5 would have been better phraseology than reject, no doubt. We must at least have the solid belief (of a committer that that has done a proper review) that a patch cannot be polished in an appropriate timeframe, I already pointed out some pretty serious problems with the updatable views patch. Are you claiming they are trivial to fix? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 3:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: We must at least have the solid belief (of a committer that that has done a proper review) that a patch cannot be polished in an appropriate timeframe, I already pointed out some pretty serious problems with the updatable views patch. Are you claiming they are trivial to fix? Not at all. I think the deferral of that particular patch is the correct thing to do because there are confirmed, real problems with it that are not realistic to fix in an appropriate timeframe for the release. The primary case that I'm objecting to is HS which you've been saying will take 10 - 12 months to complete having by your own admission not looked at the code or followed the discussion particularly closely. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, 2009-01-27 at 14:10 +, Dave Page wrote: On Tue, Jan 27, 2009 at 2:01 PM, Peter Eisentraut pete...@gmx.net wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. That will send a fine message to those companies that have sponsored development work - that we will arbitrarily reject large patches that have been worked on following the procedures that we require. We are not subject to the whims of company sponsorship. We are not a company with shareholders... Where have I heard that before? We must at least have the solid belief (of a committer that that has done a proper review) that a patch cannot be polished in an appropriate timeframe, or another justifiable reason for rejecting rather than vague handwaving, guesswork and estimates based on email traffic. If this is actually what happen then I agree. I am not sure that it is though. If we do not, we will rapidly find that no company wants to sponsor features for PostgreSQL in the future for fear that their money will be wasted even if they jump through all the right hoops. I sincerely doubt this. The sponsoring company if properly educated about the process understands this is a possibility. It is a risk. If they don't have a contract in place that allows for things like this including responsibility on the developer end, then that is there problem. Not to mention the developer could offer to support their patch for a time until it is mature enough to be committed. Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 3:48 PM, Joshua D. Drake j...@commandprompt.com wrote: On Tue, 2009-01-27 at 14:10 +, Dave Page wrote: On Tue, Jan 27, 2009 at 2:01 PM, Peter Eisentraut pete...@gmx.net wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. That will send a fine message to those companies that have sponsored development work - that we will arbitrarily reject large patches that have been worked on following the procedures that we require. We are not subject to the whims of company sponsorship. We are not a company with shareholders... Where have I heard that before? Not basing our release schedule on our commitments to shareholders is an entirely different thing to treating sponsors of major features like crap by arbitrarily bouncing the patches they've paid to have properly developed within the community process with no good reason. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, 2009-01-27 at 15:51 +, Dave Page wrote: On Tue, Jan 27, 2009 at 3:48 PM, Joshua D. Drake j...@commandprompt.com wrote: On Tue, 2009-01-27 at 14:10 +, Dave Page wrote: On Tue, Jan 27, 2009 at 2:01 PM, Peter Eisentraut pete...@gmx.net wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. That will send a fine message to those companies that have sponsored development work - that we will arbitrarily reject large patches that have been worked on following the procedures that we require. We are not subject to the whims of company sponsorship. We are not a company with shareholders... Where have I heard that before? Not basing our release schedule on our commitments to shareholders is an entirely different thing to treating sponsors of major features like crap by arbitrarily bouncing the patches they've paid to have properly developed within the community process with no good reason. Certainly but I haven't seen a suggestion to that. Updateable views has as I have seen in threads, issues that can not be fixed in the appropriately time line. If they can be fixed for 8.5 great. Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Dave Page dp...@pgadmin.org writes: On Tue, Jan 27, 2009 at 3:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: I already pointed out some pretty serious problems with the updatable views patch. Are you claiming they are trivial to fix? Not at all. I think the deferral of that particular patch is the correct thing to do because there are confirmed, real problems with it that are not realistic to fix in an appropriate timeframe for the release. The primary case that I'm objecting to is HS which you've been saying will take 10 - 12 months to complete having by your own admission not looked at the code or followed the discussion particularly closely. Well, perhaps I'm being pessimistic, or perhaps you're being optimistic. What is undeniable fact is that HS will not be committable this week, which will make it three months since feature freeze. As for when it *will* be committable --- Heikki is saying two weeks if no new problems crop up, but given the rate at which new problems have been found so far, what are the odds of that? We've seen this movie before. Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. Even if the improbable happens and there's an apparently bug-free patch ready in two weeks, I think it would be far better project management to have it go in at the beginning of the 8.5 cycle than at the tail end of 8.4. This is a major patch that has very real possibilities for breaking plain old crash recovery, even for users not using HS. The idea of shipping it with only a minimal amount of testing should scare the pants off you. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 4:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Since it's going to take us two weeks to clean up the other loose ends anyway, there's no harm in letting Simon and Heikki try to complete the patch by then. But I'll happily lay a side bet with you about what the situation will be two weeks from now. If Heikki comes back and says it's not feasible, then so be it. I'll be happy because that decision was made following a proper review and some effort to get the work done. Even if the improbable happens and there's an apparently bug-free patch ready in two weeks, I think it would be far better project management to have it go in at the beginning of the 8.5 cycle than at the tail end of 8.4. This is a major patch that has very real possibilities for breaking plain old crash recovery, even for users not using HS. The idea of shipping it with only a minimal amount of testing should scare the pants off you. It scares me for sure, but I'm reassured because I know and trust the guys at 2ndQuadrant who have been testing extensively, as well as other people such as Merlin and Heikki. Most patches get far less testing I'd wager. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, 2009-01-27 at 11:23 -0500, Tom Lane wrote: Dave Page dp...@pgadmin.org writes: On Tue, Jan 27, 2009 at 3:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: I already pointed out some pretty serious problems with the updatable views patch. Are you claiming they are trivial to fix? Even if the improbable happens and there's an apparently bug-free patch ready in two weeks, I think it would be far better project management to have it go in at the beginning of the 8.5 cycle than at the tail end of 8.4. This is a major patch that has very real possibilities for breaking plain old crash recovery, even for users not using HS. The idea of shipping it with only a minimal amount of testing should scare the pants off you. This is my take as well. This is very real, very scary things that are being worked on. HS should only ship after a very, very long non change cycle (meaning no significant bugs (or refactoring) found in HS patch for X period of time)... say after a full 8.5 dev cycle. I do not want to commit this patch and then have to yank it out 3 months from now. Lastly, the last time a developer told me two weeks it was 3 months. Unless we get a written development plan that describes specifically what, when, why and how long I am severely suspect that Heikki or Simon have a clue on an actual deliverable time line (no offense guys). Developers are generally overly optimistic about their delivery time lines. Oh and before someone says, Why would I write that. It is a waste of time when I could be coding... That was the second thing they said to be before it took 3 months. Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tuesday 27 January 2009 17:47:52 Dave Page wrote: The primary case that I'm objecting to is HS which you've been saying will take 10 - 12 months to complete having by your own admission not looked at the code or followed the discussion particularly closely. Is there another committer or expert with a different ETA estimate? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 4:42 PM, Peter Eisentraut pete...@gmx.net wrote: On Tuesday 27 January 2009 17:47:52 Dave Page wrote: The primary case that I'm objecting to is HS which you've been saying will take 10 - 12 months to complete having by your own admission not looked at the code or followed the discussion particularly closely. Is there another committer or expert with a different ETA estimate? Heikki, who has been in-depth reviewing the patch for some time, quoted a figure of 2 weeks for the known issues. AFAIK, he has not revised that estimate since. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Joshua D. Drake wrote: Lastly, the last time a developer told me two weeks it was 3 months. Unless we get a written development plan that describes specifically what, when, why and how long I am severely suspect that Heikki or Simon have a clue on an actual deliverable time line (no offense guys). Developers are generally overly optimistic about their delivery time lines. I'm totally aware of that. My two weeks estimate really was with the assumption that no new major issues crop up. I totally expect that new major issues will crop up. But I'm going to give it a shot. I'm planning to review the code anyway, whether it goes into 8.4 or 8.5, so I might as well keep going and hope for the best. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, 2009-01-27 at 18:46 +0200, Heikki Linnakangas wrote: Joshua D. Drake wrote: Lastly, the last time a developer told me two weeks it was 3 months. Unless we get a written development plan that describes specifically what, when, why and how long I am severely suspect that Heikki or Simon have a clue on an actual deliverable time line (no offense guys). Developers are generally overly optimistic about their delivery time lines. I'm totally aware of that. My two weeks estimate really was with the assumption that no new major issues crop up. I totally expect that new major issues will crop up. But I'm going to give it a shot. I'm planning to review the code anyway, whether it goes into 8.4 or 8.5, so I might as well keep going and hope for the best. Well I certainly can't argue with that :) Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Dave Page dp...@pgadmin.org writes: Not basing our release schedule on our commitments to shareholders is an entirely different thing to treating sponsors of major features like crap by arbitrarily bouncing the patches they've paid to have properly developed within the community process with no good reason. Nobody has suggested bouncing HS; there is only a debate about how soon it's likely to be appliable. Any company who imagined they had a guarantee about it getting into 8.4 is simply misguided. As for SEPostgres, I think that bouncing it entirely is quite a possible outcome, but that's because there does not appear to be adequate interest to justify taking on a major maintenance burden (and anyone who thinks it won't be a major burden is equally misguided --- at the very least it will be an endless source of bug reports that we'll be forced to classify as security issues, with all the hoop-leaping that that entails). We are not bound to accept features that are only wanted by a small number of users, no matter how badly those users want them. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 5:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: Not basing our release schedule on our commitments to shareholders is an entirely different thing to treating sponsors of major features like crap by arbitrarily bouncing the patches they've paid to have properly developed within the community process with no good reason. Nobody has suggested bouncing HS; there is only a debate about how soon it's likely to be appliable. Any company who imagined they had a guarantee about it getting into 8.4 is simply misguided. I was complaining about it being bounced to 8.5 without proper review as I've said elsewhere in one of these threads. As for SEPostgres, I think that bouncing it entirely is quite a possible outcome, but that's because there does not appear to be adequate interest to justify taking on a major maintenance burden (and anyone who thinks it won't be a major burden is equally misguided --- at the very least it will be an endless source of bug reports that we'll be forced to classify as security issues, with all the hoop-leaping that that entails). We are not bound to accept features that are only wanted by a small number of users, no matter how badly those users want them. That wouldn't be 'arbitrarily bounced' though, it would be with good reason. And I have no problem with that. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Dave Page dp...@pgadmin.org writes: On Tue, Jan 27, 2009 at 5:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Nobody has suggested bouncing HS; there is only a debate about how soon it's likely to be appliable. Any company who imagined they had a guarantee about it getting into 8.4 is simply misguided. I was complaining about it being bounced to 8.5 without proper review as I've said elsewhere in one of these threads. Huh? It's been getting proper review for three months now. I don't think I need to have studied the code in detail before I'm entitled to point out that it's horribly late and is still getting revised like mad. If it weren't something that people desperately want, it would have been punted to 8.5 quite some time ago. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, 2009-01-27 at 16:01 +0200, Peter Eisentraut wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. BTree-GIN has been ready for committer review for quite some time. It has been mostly-ready for much longer: the only real code change since submission was a workaround to support numeric that I requested. It's also a small patch, easy to review, and offers a nice benefit. Kenneth Marshall's updated hash functions (separated mix()/final()) haven't had any code changes since Nov. 4, the code is tiny, and the only thing we've been waiting on is proof that it doesn't hurt randomness. Ken has put in the effort to show that. At least look at his analysis, and see if you agree. I think both of these deserve at least a glance by a committer before bouncing them. There are more committers who can look at these features than, say, Hot Standby or Sync Rep. Details on commitfest wiki for those interested. GIN Fast Insert is also ready for a committer review. I happen to like this feature, and I think it is compelling for anyone using GIN (Teodor posted some good performance results today). However, it has gone through more code changes than the two patches above and it's more complex, so if you think this deserves to be put off, that's understandable. I think it would be worth a look for any committer not on the critical path for release though. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, Jan 27, 2009 at 1:50 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2009-01-27 at 16:01 +0200, Peter Eisentraut wrote: Updatable views is reverted. I agree that we should reject the rest and prepare a release. BTree-GIN has been ready for committer review for quite some time. It [...] Kenneth Marshall's updated hash functions (separated mix()/final()) [] I think both of these deserve at least a glance by a committer before bouncing them. While we're at it, I think the Ramon Lawrence/Bryce Cutt patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets also deserves a look. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2009 at 1:50 PM, Jeff Davis pg...@j-davis.com wrote: I think both of these deserve at least a glance by a committer before bouncing them. While we're at it, I think the Ramon Lawrence/Bryce Cutt patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets also deserves a look. There is no proposal or intention to bounce any of the remaining commitfest items without consideration. SEPostgres and Hot Standby are the ones under debate. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Tue, 2009-01-27 at 15:09 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2009 at 1:50 PM, Jeff Davis pg...@j-davis.com wrote: I think both of these deserve at least a glance by a committer before bouncing them. While we're at it, I think the Ramon Lawrence/Bryce Cutt patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets also deserves a look. There is no proposal or intention to bounce any of the remaining commitfest items without consideration. SEPostgres and Hot Standby are the ones under debate. I must have misunderstood Peter's comment. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Sun, 2009-01-25 at 12:06 -0500, Tom Lane wrote: If we want to ensure that 8.5 development opens soon, what we have to do is reject those two patches, revert updatable views, and finish up the other stuff (which is all small and could likely be dealt with in a week or two). That puts us in position to go beta by perhaps mid-February with release perhaps on May 1. If we don't, I hereby predict that 8.4 release will not happen before September. Trying to deal with those late, large features will add *at least* one more month to commitfest and *at least* one more month to beta (you think they'll be bug-free?). As our new president has been reminding us, it's time to start making hard choices. We're clearly ahead of where we were in 8.3. Any release with big features in it will take longer, whether you wait a year, or not. I think the choice here is currently between features or release schedule, but we could look at things differently and generate some other options. What would our users want? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Sun, 2009-01-25 at 12:06 -0500, Tom Lane wrote: If we want to ensure that 8.5 development opens soon, what we have to do is reject those two patches, revert updatable views, and finish up the other stuff (which is all small and could likely be dealt with in a week or two). That puts us in position to go beta by perhaps mid-February with release perhaps on May 1. If we don't, I hereby predict that 8.4 release will not happen before September. Expecting a release around September with those patches a bit optimistic, IMHO. We already experienced that a release just after summer is not possible. So, unless there is a way to commit SELinux+Hot Standby+Sync Replication in the next two weeks (or so), as you wrote, we should reject those patches and release 8.4 on time. (Oh, personally I'd like to see all these features at 8.4, from advocacy side). -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org signature.asc Description: This is a digitally signed message part
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Simon Riggs wrote: On Sun, 2009-01-25 at 12:06 -0500, Tom Lane wrote: Any release with big features in it will take longer, whether you wait a year, or not. Well, big features that land early in the release cycle don't delay the release. Just the ones that are submitted in the last commit fest. I think the choice here is currently between features or release schedule, but we could look at things differently and generate some other options. Yeah, that's the tradeoff. features vs. release schedule. I don't see how else you could look at things. What would our users want? I'm sure it depends on the user. Users that are more interested in the features we already have in the bag like window functions and WITH-clause, will obviously prefer to release earlier without hot standby. And users that want hot standby (or SE-postgresql) will prefer to delay the release and have those features included. I don't personally have an opinion, because I don't work with any real-world databases. It's a tradeoff and both options seem good to me. I'm going to stick around and keep reviewing Hot Standby as long as it takes, and when it's ready I can commit it to either 8.4 or 8.5. It's the same amount of work for me either way. There's still a list of non-resolved issues that I estimate will take about two weeks to address. And there's a good possibility that new issues arise, which require yet more time. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Mon, Jan 26, 2009 at 11:35 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I'm sure it depends on the user. Users that are more interested in the features we already have in the bag like window functions and WITH-clause, will obviously prefer to release earlier without hot standby. And users that want hot standby (or SE-postgresql) will prefer to delay the release and have those features included. At LinuxLive (UK) the overwhelming majority of people I spoke to over three days wanted hot standby and replication (preferably multi-master, but thats another story). Window functions, recursive queries, SE PostgreSQL, updatable views and other new features were barely mentioned. There's still a list of non-resolved issues that I estimate will take about two weeks to address. And there's a good possibility that new issues arise, which require yet more time. That doesn't seem like it's worth deferring such a useful feature for 12+ months for - especially as the work is clearly ongoing and can happen in parallel with work on other patches. As I've pointed out before, we're not a commercial company working for our shareholders, we're a FOSS project working for our end users. If we can include an important and popular feature like this at the expense of a few weeks extra wait for the release, it seems to me that we'll be serving our users far better overall than making a fair percentage of them wait another 12 months for work that is more or less complete. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Sun, Jan 25, 2009 at 12:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Particularly with regard to hot standby, which by any sane reading was not close to being committable on 1 November (a fortiori from the fact that it's *still* not committable despite large amounts of later work). While I haven't follwed every detail of this patch set, I'm not quite sure I see that as being very fair to Simon. Simon has put a lot of time into Hot Standby and has followed the pseudo-defacto community process from design through what he believes to be near-completion; he can't be sure of completion until someone reviews his work. Honestly, I'm not trying to marginalize your effort reviewing other patches, and I know everyone is busy. Hell, as much as I'd love to have this feature, I too have been unable to find enough time to review Simon's stuff. However, over the past few months, I seem to recall seeing Simon submit countless requests for review and regardless of whether it was completely ready to go November 1st or not, at this time I don't think anyone but Simon has a complete view of what his patch(es) even look like. Yet, albeit with almost no review from the committers, Simon has continually worked through testing, revising his patches, and requesting information and suggestions from the community. Frankly Tom, I don't know of anyone in the community with as much experience in the recovery code as you and Simon. So, any of the major edge case problems will probably only be found by you regardless of how many of us review Simon's work. I do know that this is a feature which a large number of Postgres users really want and were counting on being in 8.4. Looking forward, if no one wanted to review these patches in November, and seemingly no one wants to review them now, how can we expect this to change for 8.5? Can anyone point out something Simon did wrong in this process? -- Jonah H. Harris, Senior DBA myYearbook.com
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Sun, 2009-01-25 at 12:06 -0500, Tom Lane wrote: Particularly with regard to hot standby, which by any sane reading was not close to being committable on 1 November (a fortiori from the fact that it's *still* not committable despite large amounts of later work). I've wasted much time in two main areas in recent months: The first area I've wasted time on is patch review. Had I known that some patches on the queue would not be properly reviewed and that time was going to be a factor then I would not have wasted a single second on patch review. My main review item has been Sync Replication, which I did because of Core's statement that we would include that feature in this release. It would appear that my time on that has been both pointless and damaging to my own situation. The second is in developing an answer to the FATAL errors problem, which I asked a direct question on and received a very specific answer: http://archives.postgresql.org/pgsql-hackers/2008-09/msg01809.php I spent two weeks working out a difficult solution to that, a week subsequently arguing with Heikki (possibly longer) about whether it was necessary and then two weeks refactoring it back out again when you were silent. From my side, I've spent time and money supporting your views and decisions, without argument with you, though most would not see that. I learned on Jan 12 that you'd been too busy to consider this work and that a priori you argued for patch rejection. It would be much simpler if I had badly screwed up or if we'd found an awkward showstopper. That hasn't happened. In fact, I've done the full dance so far. I fail to see how rejecting unreviewed patches provides benefit for users, developers or sponsors. How does de-committing from features voted most-popular or promised by core sit with users? How will developers react when they know that helping in patch review assists their own demise? How will sponsors react when they see another set of patches rejected? If there is a problem with timing it's because we need more planning, scheduling and prioritisation of resources. But it isn't possible to do it retrospectively without causing problems, and those problems may be worse than slipping a month, or three. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Simon has put a lot of time into Hot Standby and has followed the pseudo-defacto community process from design through what he believes to be near-completion; he can't be sure of completion until someone reviews his work. I think this is a fair critique. Yet, albeit with almost no review from the committers, Simon has continually worked through testing, revising his patches, and requesting information and suggestions from the community. There really was not a lot of review from mid-October through the end of the year. That is partly because Simon was out of commission for about three weeks and did not respond (in particular) to several requests to separate ICfR from HS. That having been said, since this is such an important feature to so many people, I think it would have been better if more effort could have been put into doing what additional reviewing was still possible. However, since the turn of the year, it looks to me like Heikki has actually put quite a bit of time into reviewing and responding to issues. Still, I agree that if there's anything we should be putting our effort into as a community right now, it's this feature. If we got Hot Standby in the next release and everything else in the CommitFest got bumped, I think a lot of people would consider that a good trade (though probably not the authors of the patches that got bumped). For example, I would much rather see Tom revert the updatable views patch and work on this than spend the next week fixing updatable views. At a minimum, I think the following patches from the CommitFest wiki should be returned with feedback or rejected: 1. SE-PostgreSQL. We handled this one badly, but there's not enough time to fix it now. 8.5. 2. rmgr hooks and contrib/rmgr_hook. Reject because Tom and Heikki don't believe it's the right approach. Need better use cases. 3. Synchronous log-shipping replication. We handled this one well, but it's not in good enough shape. 8.5. 4. pg_upgrade script. I haven't heard much about this in a while... I am doubtful that it is production-quality, but maybe I'm wrong? 5. Reducing some DDL Locks to ShareLock. No activity in a long time, no time to wait for this to be finished. 8.5. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Mon, 2009-01-26 at 13:35 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Sun, 2009-01-25 at 12:06 -0500, Tom Lane wrote: Any release with big features in it will take longer, whether you wait a year, or not. Well, big features that land early in the release cycle don't delay the release. Just the ones that are submitted in the last commit fest. Has that ever happened? :-) I don't think its chance we get big patches in last commit fest. No sponsor I ever met was happy to both sponsor and to wait. The two seem mutually exclusive. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Simon Riggs wrote: On Mon, 2009-01-26 at 13:35 +0200, Heikki Linnakangas wrote: Well, big features that land early in the release cycle don't delay the release. Just the ones that are submitted in the last commit fest. Has that ever happened? :-) I don't think its chance we get big patches in last commit fest. No sponsor I ever met was happy to both sponsor and to wait. The two seem mutually exclusive. How come it works for Linux? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Jonah H. Harris wrote: Looking forward, if no one wanted to review these patches in November, I did, and many others were active in the discussion too. and seemingly no one wants to review them now, I do. Thank you for your appreciation :-(. how can we expect this to change for 8.5? Can anyone point out something Simon did wrong in this process? Not really, except maybe started 6 months too late. Big patches simply take a long time to mature. Looking back at the timeline for hot standby, it doesn't look unreasonable at all: September: First discussion about the user-visible behavior, transaction isolation etc. Big architectural decisions are made, like where that snapshots are taken. Infrastructure changes for recovery patch is reviewed, and pushed to next commitfest because of issues. Discussion started on how to handle (heavy-weight) locks. October: Discussion on various administration commands, how to handle b-tree splits, and some other stuff. More discussion on how to derive snapshots. First complete hot standby patch is submitted. A patch to change the way subtransactions is submitted, and committed. November: Various people test and review the patch, including Mark Kirkwood and Pavan Deolasee. Bugs are found and fixed. Decision is made that SIGINT should kill an idle-in-transaction transaction as well. December: Discussion on slotids and the B-tree killed items problem. January: Major changes; slotids eliminated, conflict resolution mechanism rewritten. RestoreBkpBlocks refactoring committed. More bugs in conflict resolution unearthed. Discussion on adding a GUC. There has been steady progress, starting from basic design discussions and decisions, moving on to implementation details. Progress never stalled for a long time. I can see no wrongdoing on Simon's part, and there is also no grounds to say that the community has neglected this patch. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Robert Haas robertmh...@gmail.com wrote: Still, I agree that if there's anything we should be putting our effort into as a community right now, it's this feature. If we got Hot Standby in the next release and everything else in the CommitFest got bumped, I think a lot of people would consider that a good trade Based on discussions at Milwaukee BarCamp, improvements in replication would do more to break down barriers to migration to PostgreSQL than anything else. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Robert Haas wrote: At a minimum, I think the following patches from the CommitFest wiki should be returned with feedback or rejected: 1. SE-PostgreSQL. We handled this one badly, but there's not enough time to fix it now. 8.5. Unacceptable. Please make it clear how many items should be fixed/reworked before rejecting it due to the lack of time, at least. (I'm optimistic for its design and quality.) In addition, why does it impossible to be worked in parallel? Maybe, I cannot contribute Simon's work, but I can fix/rework for SE-PostgreSQL in parallel, with my highest priority. As I noted in another message, the current trend strongly requires availability and security for the platform of SaaS/PaaS or cloud-computing. So, we should also include SE-PostgreSQL within v8.4. I believe it makes PostgreSQL more widespreadly applied. 2. rmgr hooks and contrib/rmgr_hook. Reject because Tom and Heikki don't believe it's the right approach. Need better use cases. 3. Synchronous log-shipping replication. We handled this one well, but it's not in good enough shape. 8.5. IMO, this feature is also key factor for availablity. If author can have enough activity, we should not give up right now... 4. pg_upgrade script. I haven't heard much about this in a while... I am doubtful that it is production-quality, but maybe I'm wrong? 5. Reducing some DDL Locks to ShareLock. No activity in a long time, no time to wait for this to be finished. 8.5. ...Robert -- KaiGai Kohei kai...@kaigai.gr.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Mon, 2009-01-26 at 12:05 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Mon, 2009-01-26 at 13:35 +0200, Heikki Linnakangas wrote: Well, big features that land early in the release cycle don't delay the release. Just the ones that are submitted in the last commit fest. Has that ever happened? :-) I don't think its chance we get big patches in last commit fest. No sponsor I ever met was happy to both sponsor and to wait. The two seem mutually exclusive. How come it works for Linux? Not sure. How come up it happens here is a more relevant question. I can only tell you that people only get their cheque books out when they are certain somebody else will not do it for them for free. This happened to you also, or did you have this terrible desire for CRC checks on data blocks quite late in release cycle? If there hadn't been blockers you'd be being lightly barbecued on the patch queue also. :-) -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Jonah H. Harris wrote: how can we expect this to change for 8.5? Can anyone point out something Simon did wrong in this process? Not really, except maybe started 6 months too late. Big patches simply take a long time to mature. Yeah, exactly. When we decided at PgCon that we needed to accept replication into the core, we thought that there was a realistic chance of it being in 8.4 if full-tilt development started *immediately*. For one reason or another nothing much happened until September. Simon's certainly been working hard on it since then, but it just takes longer than four months for something like this to happen. The alternatives that we realistically have got are: * ship 8.4 now (ie before the summer) without HS, plan for HS in 8.5 * delay 8.4 to late summer and ship it with an unpolished HS feature * delay 8.4 long enough to have a stable, polished HS feature; I claim this will mean it's not out before Christmas. I think anyone who thinks alternatives 2 or 3 will happen faster than sketched here is living in dreamland. So my feeling is that prudent project management would suggest that HS go into 8.5 near the beginning of that development cycle, rather than trying to push it into 8.4. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Bruce Momjian br...@momjian.us writes: Robert Haas wrote: Yeah... I'm not sure what to do about that, but as Tom pointed out, it has the disadvantage that all of these massive changes are getting put into the tree just before we start beta. Well, it is less a problem than in previous releases, so things are getting better. Are they? If we stay in commitfest mode until every single thing on the list has been dealt with, I am not sure this release cycle will be any shorter than 8.3's was. We said back at the beginning that The hope here is that we will not have enormous, previously unreviewed patches landing on us at the end of October --- if that happens, we'll be back in the same position we were in at 8.3 feature freeze. Although this schedule allows for the final commit fest to take a good deal of time, we'll reserve the right to reject patches that are too large to be reviewed in a timely fashion. We want to encourage people to do development of large features in an incremental fashion, with a new increment landing during each commit fest. and I'm beginning to think that we need to invoke that provision. Particularly with regard to hot standby, which by any sane reading was not close to being committable on 1 November (a fortiori from the fact that it's *still* not committable despite large amounts of later work). I'm also feeling that we are not in a position to commit SE-Postgres in a timely fashion; which is not KaiGai-san's fault, rather that of the community which has taken nearly zero interest in that patch. If we want to ensure that 8.5 development opens soon, what we have to do is reject those two patches, revert updatable views, and finish up the other stuff (which is all small and could likely be dealt with in a week or two). That puts us in position to go beta by perhaps mid-February with release perhaps on May 1. If we don't, I hereby predict that 8.4 release will not happen before September. Trying to deal with those late, large features will add *at least* one more month to commitfest and *at least* one more month to beta (you think they'll be bug-free?). As our new president has been reminding us, it's time to start making hard choices. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
On Sun, Jan 25, 2009 at 12:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: and I'm beginning to think that we need to invoke that provision. Particularly with regard to hot standby, which by any sane reading was not close to being committable on 1 November (a fortiori from the fact that it's *still* not committable despite large amounts of later work). maybe what we need is to put a policy for large patches... something like: if you submit a large patch that introduce fairly complex new features very late in the release cycle (maybe at the last commit fest or previous to that one) then there are no promises about committers to review them... maybe that will enforce authors to send patches more often and more early... and of course, the rest of us that make some kind of review (call it: testing or code review) should start making those reviews on large patches (just like someone suggests) -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Tom Lane wrote: and I'm beginning to think that we need to invoke that provision. Particularly with regard to hot standby, which by any sane reading was not close to being committable on 1 November (a fortiori from the fact that it's *still* not committable despite large amounts of later work). I'm also feeling that we are not in a position to commit SE-Postgres in a timely fashion; which is not KaiGai-san's fault, rather that of the community which has taken nearly zero interest in that patch. Yes, few feedbacks have been a concern for me, though I can understand that reviewers/commiters had to deal with various kind of many patches for the v8.4 development cycle. (Needless to say, I remember Tom commented a lot on CommitFest:May and it made SE-PostgreSQL well.) If SE-PostgreSQL is not still committable phase, I want to discuss what code should be reworked and what items are remained. Because it was unclear, I had to review patches by myself in this January. :( In addition, I would like to mention about the scale of patches. | % diffstat sepostgresql-sepgsql-8.4devel-3-r1467.patch says, | 110 files changed, 9813 insertions(+), 16 deletions(-), 924 modifications(!) However, the total number of insertions under src/backend/security and src/include/security is 8207 lines of 9813. It modifies 924 lines, but 504 lines of 924 is due to a new field of pg_attribute system catalog. Please note that it never applies massive rough-mannered fixes all over the core implementation, so it is informidable. If we want to ensure that 8.5 development opens soon, what we have to do is reject those two patches, revert updatable views, and finish up the other stuff (which is all small and could likely be dealt with in a week or two). That puts us in position to go beta by perhaps mid-February with release perhaps on May 1. If we don't, I hereby predict that 8.4 release will not happen before September. Trying to deal with those late, large features will add *at least* one more month to commitfest and *at least* one more month to beta (you think they'll be bug-free?). I can understand we cannot postpone to release v8.4 forever and it is important to see a calendar. However, I would like folks to see not only a calendar, but a current *trend* also. Nowaday, we have no days without looking a term such as SaaS/PaaS or cloud-computing. It consists of various kind of web applications and highly-integrated database system on cluster systems. In this movement, end-users pay their attention on availability and security most. If we can include these features in a timely fashion, it makes PostgreSQL more attractive compared to other DBMSs. Needless to say, I'll pay my maximum effort to reduce the additional days, even if it is estimated one more month is necessary. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: 8.4 release planning (was Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules)
Robert Haas wrote: I would, however, like to see us make a commitment to actually review SE-PostgreSQL. There was some talk that we might not want to include this feature in core at all, and if that is the case then I think it is long past time to make that decision. Assuming that isn't the case, then we need to get past the stage where we make occasional comments on the overall architecture and get down to really reading the code. I am willing to help with this but I don't have either the time or the qualifications to do it single-handedly. To be brutally honest, I don't care about the feature at all: the only thing I ever do with SELinux is turn it off (row-level DAC is mildly interesting to me). But I think that if we want to build a community of developers around PostgreSQL, we'd better at least look at the work they submit. I would like to call for SELinux folks to join reviewing SE-PostgreSQL from the point of view of security expert. If folks in pgsql-hackers have questions, I belive they can provide an answer to the questions. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
