Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 12:03:05AM +0200, Dimitri Fontaine wrote: > Hi, > > Peter Eisentraut writes: > > On ons, 2009-08-26 at 14:26 -0400, Robert Haas wrote: > >> Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months > >> doesn't necessarily buy you much, either. I'm good at focused > >> activity - but there was nothing focused about 8.4 beta that I could > >> see. Maybe we need some kind of TestFest process. > > > > Yeah, exactly. I can't imagine end users would know what to do during > > beta. Even assuming that you have release notes at the beginning of > > beta, you can't expect people to go through every item and do a formal > > test for it. Surely it's been tested before, else it would not be in > > the release, right? > > Well we all know that developpers are really bad at testing, because > they tend to test for cases they though about while developping the code > rather than being creative and running a full application against the > overall new product. > > Now, it could be that what we miss is some tool suite to enable > application people to test their full code against our new releases and > check results, performance, plans, etc. > > I know about a couple of tools to get there, Tsung and Playr. And the > focus is performance testing and scalability, not that it still works. > > Is the offering good enough? We might need to run some kind of tutorials > for users to be able to run large tests easily, and maybe think about > some newer tools allowing to compare logs of two application runs in two > database versions (capture all queries and result in a database, then > have a way to diff). Then beta testing would mean having a spare machine > where to run the magic regression test suite against some internal > application. Someone else upthread mentioned it as well; having an easy way to capture queries for a day and replaying it towards a server with the beta software would be a totally awesome way to beta test. Query results would be compared and any anomalies reported. Doing this in realtime over both servers would be an even more appealing option (at least to me and my environment). I could easily capture queries from a dev machine (where development is still towards a stable release of postgres) and test them on the new Beta release. I haven't looked at either Tsung or Playr (but I will), so if anyone knows if this is possible with these tools or perhaps is already doing it, it would be nice with a short "tutorial" (documented on the wiki?) on steps on how to setup such an environment. Kind regards, Kristian. -- Kristian LarssonKLL-RIPE +46 704 264511k...@spritelink.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Josh Berkus wrote: > Bruce, > > > I am not sure what other checklist items there would be (or I am > > refusing to divulge). > > Hopefully the last is a joke. ;-) Yes. > So, the only post-CF tasks are issues with specific patches? This seems > resolvable, especially if we take a hard line with patch readiness. > There isn't anything else in that period? Nope. Release notes and open items is all I remember, and the release notes were done at the end of the commit-fest, so that isn't really even an issue. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Bruce, > I am not sure what other checklist items there would be (or I am > refusing to divulge). Hopefully the last is a joke. ;-) So, the only post-CF tasks are issues with specific patches? This seems resolvable, especially if we take a hard line with patch readiness. There isn't anything else in that period? This doesn't sound that difficult then. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Kevin Grittner wrote: > Bruce Momjian wrote: > > > The issues are different for every commitfest-beta period, so I have > > no idea what to list there, but we do alway have an open issues wiki > > that is maintained, at least for the most recent releases. > > After a quick search of the wiki, it appears that the list for 8.4 > was: > > http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items > > and that there is not yet a list for 8.5. Is that correct? > > If I understand what you're saying, this list would contain issues > where a patch was committed and later found to have problems which > need to be fixed. Did I understand that correctly? Anything else go > on there, or possibly belong on there? Can we take the absence of a > list for 8.5 to indicate that no such problems have been found with > any patches committed since 8.4 was tagged? Yes, though I have a few items that I should transfer from my mailbox to that list. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Kevin Grittner wrote: > Bruce Momjian wrote: > > > it gets no easier to make the decisions later rather than now. The > > delay forces us to make a final decision. We often had months to > > make the decision earlier, but didn't. > > So you're advocating that we find a way to force more timely > decisions? That would be good. :-) -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On sön, 2009-08-30 at 21:09 -0400, Robert Haas wrote: > I really can't understand why it isn't possible for us to find a way > to make an annual release happen, and with more than 8-12 weeks of > development time (ie non-CommitFest non-beta) available during that > year. I understand that there is a need for some time to be set aside > for reviewing, testing, debugging, packaging, and so forth, but the > current schedule contemplates that this time is upwards of 75%, and I > think that's excessive. Well, the best way to improve that is to organize people and push things forward when the time comes. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Bruce Momjian wrote: > it gets no easier to make the decisions later rather than now. The > delay forces us to make a final decision. We often had months to > make the decision earlier, but didn't. So you're advocating that we find a way to force more timely decisions? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Bruce Momjian wrote: > The issues are different for every commitfest-beta period, so I have > no idea what to list there, but we do alway have an open issues wiki > that is maintained, at least for the most recent releases. After a quick search of the wiki, it appears that the list for 8.4 was: http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items and that there is not yet a list for 8.5. Is that correct? If I understand what you're saying, this list would contain issues where a patch was committed and later found to have problems which need to be fixed. Did I understand that correctly? Anything else go on there, or possibly belong on there? Can we take the absence of a list for 8.5 to indicate that no such problems have been found with any patches committed since 8.4 was tagged? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Mon, Aug 31, 2009 at 2:20 PM, Bruce Momjian wrote: > Knowing about the problem usually isn't hard , e.g. \df, but getting > agreement on them is. One nifty idea would be to do a commit-fest for > open items so we can get to beta. I like that idea very much. > The last commit-fest usually is long > because we can't postpone patches easily and often we are not 100% sure > how to apply them either, so that make it extra-long. > > I am not sure what other checklist items there would be (or I am > refusing to divulge). LOL. Well, there are things like release notes... and maybe others? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Josh Berkus wrote: > Bruce, > > > Huh, who has asked for a list from me? This entire post is mostly > > over-the-top and not worth responding to. > > To quote myself: > > > Post-CF: > > Make a list (now, not in January) of the tasks which need to be done > > between CFend and Beta. We'll find that some of them could be done by > > someone other than Tom and Bruce, and that others could be done before > > CFend. > > > > Beta: > > Create a mailing list (why don't we have a list for testers? is > > testing less important than the JDBC driver?) and a simple web app or > > templated wiki page for testers. Allow people to check in with which > > version they tested (Alpha1,2,3, Beta 1,2,3) and what tests they ran, > > and issues encountered (if any). We should do this now so we can get > > started with Alpha testing. > > When this testing gets into swing, we can also look at recruiting > > volunteers to run various popular OSS apps' test suites against the > > developing version of PostgreSQL. > > Once beta starts, have a list of pending issues in some > > editable/commentable location (wiki is ok for this, or we can pervert > > the commitfest app) *as soon as those issues arise* so that as many > > hackers as possible can work on those issues. We did do a "pending > > issues" list for 8.4, but not until we were already over a month into > > beta, and then the list wasn't very accurate. > > Therefore: > > I will create a "cleanup issues" wikipage. Please contribute to it by > listing the *general* kinds of things you need to do between CF and > beta. Then we can look at which things don't need your special > experience and could be done by other contributors. That was a request for me to answer? I had no idea. The issues are different for every commitfest-beta period, so I have no idea what to list there, but we do alway have an open issues wiki that is maintained, at least for the most recent releases. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Joshua D. Drake wrote: > On Mon, 2009-08-31 at 13:59 -0400, Bruce Momjian wrote: > > Robert Haas wrote: > > > On Mon, Aug 31, 2009 at 1:57 PM, Bruce Momjian wrote: > > > > Josh Berkus wrote: > > > >> Per the above, it would not. ?It would make things worse. ?This has > > > >> been > > > >> true at every other OSS project I've seen documented (disastrously so > > > >> with MySQL); there is no reason to believe that Postgres would be any > > > >> different. > > > >> > > > >> I also do not see why you are so resistant to the idea of documenting a > > > >> tracking the post-CF steps so that we can get more people on them. > > > > > > > > Huh, who has asked for a list from me? ?This entire post is mostly > > > > over-the-top and not worth responding to. > > > > > > OK, I so request. :-) > > > > What do you want to know? Would someone post exactly what question I > > have not answered in the past? > > This is a fair point. I bet 10 bucks that a lot of the questions that > would be asked would be answered with, "check the archives". > > Didn't we do a release wiki page at one point? Yes, we have a wiki for open items for the current major release. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas wrote: > That having been said, I think there is a legitimate concern about > organizing and documenting the steps that are required to get a > release out the door. A number of people have said (on this thread > and previous ones) that we didn't know what we were supposed to be > doing during the period after the end of the last CommitFest and prior > to release. It appeared that all of the activity (to the extent that > there was any activity) was by committers, particularly you and Tom. > > Well, OK. We want the release to happen faster next time. We're > willing to help. In order to help, we first need a list of the tasks > that need to be completed after the last CommitFest and before > release. Then we can try to figure out whether any of those tasks can > be done (or assisted with) by someone other than you or Tom. > > Can you provide one? Well, at the end of the release I have a mailbox full of open items, that I think need to be addressed before we go into beta. Tom has a similar list. I usually put my mbox file up on a web site, and sometimes it is transfered to a wiki by others. Knowing about the problem usually isn't hard , e.g. \df, but getting agreement on them is. One nifty idea would be to do a commit-fest for open items so we can get to beta. The last commit-fest usually is long because we can't postpone patches easily and often we are not 100% sure how to apply them either, so that make it extra-long. I am not sure what other checklist items there would be (or I am refusing to divulge). -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Bruce, > Huh, who has asked for a list from me? This entire post is mostly > over-the-top and not worth responding to. To quote myself: > Post-CF: > Make a list (now, not in January) of the tasks which need to be done > between CFend and Beta. We'll find that some of them could be done by > someone other than Tom and Bruce, and that others could be done before > CFend. > > Beta: > Create a mailing list (why don't we have a list for testers? is > testing less important than the JDBC driver?) and a simple web app or > templated wiki page for testers. Allow people to check in with which > version they tested (Alpha1,2,3, Beta 1,2,3) and what tests they ran, > and issues encountered (if any). We should do this now so we can get > started with Alpha testing. > When this testing gets into swing, we can also look at recruiting > volunteers to run various popular OSS apps' test suites against the > developing version of PostgreSQL. > Once beta starts, have a list of pending issues in some > editable/commentable location (wiki is ok for this, or we can pervert > the commitfest app) *as soon as those issues arise* so that as many > hackers as possible can work on those issues. We did do a "pending > issues" list for 8.4, but not until we were already over a month into > beta, and then the list wasn't very accurate. Therefore: I will create a "cleanup issues" wikipage. Please contribute to it by listing the *general* kinds of things you need to do between CF and beta. Then we can look at which things don't need your special experience and could be done by other contributors. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Mon, 2009-08-31 at 13:59 -0400, Bruce Momjian wrote: > Robert Haas wrote: > > On Mon, Aug 31, 2009 at 1:57 PM, Bruce Momjian wrote: > > > Josh Berkus wrote: > > >> Per the above, it would not. ?It would make things worse. ?This has been > > >> true at every other OSS project I've seen documented (disastrously so > > >> with MySQL); there is no reason to believe that Postgres would be any > > >> different. > > >> > > >> I also do not see why you are so resistant to the idea of documenting a > > >> tracking the post-CF steps so that we can get more people on them. > > > > > > Huh, who has asked for a list from me? ?This entire post is mostly > > > over-the-top and not worth responding to. > > > > OK, I so request. :-) > > What do you want to know? Would someone post exactly what question I > have not answered in the past? This is a fair point. I bet 10 bucks that a lot of the questions that would be asked would be answered with, "check the archives". Didn't we do a release wiki page at one point? Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Mon, Aug 31, 2009 at 1:59 PM, Bruce Momjian wrote: > Robert Haas wrote: >> On Mon, Aug 31, 2009 at 1:57 PM, Bruce Momjian wrote: >> > Josh Berkus wrote: >> >> Per the above, it would not. ?It would make things worse. ?This has been >> >> true at every other OSS project I've seen documented (disastrously so >> >> with MySQL); there is no reason to believe that Postgres would be any >> >> different. >> >> >> >> I also do not see why you are so resistant to the idea of documenting a >> >> tracking the post-CF steps so that we can get more people on them. >> > >> > Huh, who has asked for a list from me? ?This entire post is mostly >> > over-the-top and not worth responding to. >> >> OK, I so request. :-) > > What do you want to know? Would someone post exactly what question I > have not answered in the past? I don't know whether there is a specific question that you have refused to answer in the past, or not. My suspicion is that there isn't, but perhaps someone else is aware of something I'm not. That having been said, I think there is a legitimate concern about organizing and documenting the steps that are required to get a release out the door. A number of people have said (on this thread and previous ones) that we didn't know what we were supposed to be doing during the period after the end of the last CommitFest and prior to release. It appeared that all of the activity (to the extent that there was any activity) was by committers, particularly you and Tom. Well, OK. We want the release to happen faster next time. We're willing to help. In order to help, we first need a list of the tasks that need to be completed after the last CommitFest and before release. Then we can try to figure out whether any of those tasks can be done (or assisted with) by someone other than you or Tom. Can you provide one? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas wrote: > On Mon, Aug 31, 2009 at 1:57 PM, Bruce Momjian wrote: > > Josh Berkus wrote: > >> Per the above, it would not. ?It would make things worse. ?This has been > >> true at every other OSS project I've seen documented (disastrously so > >> with MySQL); there is no reason to believe that Postgres would be any > >> different. > >> > >> I also do not see why you are so resistant to the idea of documenting a > >> tracking the post-CF steps so that we can get more people on them. > > > > Huh, who has asked for a list from me? ?This entire post is mostly > > over-the-top and not worth responding to. > > OK, I so request. :-) What do you want to know? Would someone post exactly what question I have not answered in the past? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Kevin Grittner wrote: > Bruce Momjian wrote: > > > Yep, the bottom line here is that patches get into CVS, but issues > > come up related to the patch, and we keep looking for good fixes, > > but once the final commit-fest is over, we _have_ to fix these > > issues. > > If, hypothetically, it might hold up the release for two weeks while > such issues are sorted out, might it be better to revert and say the > patch missed the release because it wasn't workable enough at the end > of the last CF to allow a beta release to be generated? If the net > result was that a feature or two were delayed until the next release, > but all developers had two more weeks of development time in the next > release cycle, it seems like reverting would be a net gain. The problem is that many of these decisions are complex so it gets no easier to make the decisions later rather than now. The delay forces us to make a final decision. We often had months to make the decision earlier, but didn't. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Mon, Aug 31, 2009 at 1:57 PM, Bruce Momjian wrote: > Josh Berkus wrote: >> Per the above, it would not. It would make things worse. This has been >> true at every other OSS project I've seen documented (disastrously so >> with MySQL); there is no reason to believe that Postgres would be any >> different. >> >> I also do not see why you are so resistant to the idea of documenting a >> tracking the post-CF steps so that we can get more people on them. > > Huh, who has asked for a list from me? This entire post is mostly > over-the-top and not worth responding to. OK, I so request. :-) ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Josh Berkus wrote: > Per the above, it would not. It would make things worse. This has been > true at every other OSS project I've seen documented (disastrously so > with MySQL); there is no reason to believe that Postgres would be any > different. > > I also do not see why you are so resistant to the idea of documenting a > tracking the post-CF steps so that we can get more people on them. Huh, who has asked for a list from me? This entire post is mostly over-the-top and not worth responding to. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Bruce Momjian wrote: > Yep, the bottom line here is that patches get into CVS, but issues > come up related to the patch, and we keep looking for good fixes, > but once the final commit-fest is over, we _have_ to fix these > issues. If, hypothetically, it might hold up the release for two weeks while such issues are sorted out, might it be better to revert and say the patch missed the release because it wasn't workable enough at the end of the last CF to allow a beta release to be generated? If the net result was that a feature or two were delayed until the next release, but all developers had two more weeks of development time in the next release cycle, it seems like reverting would be a net gain. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Mon, 2009-08-31 at 10:30 -0700, Josh Berkus wrote: > >>> Another solution would be to make major releases less frequent. > >> That's not a solution and you know it. > > > > I do? > > Ok, here's the reasons it's not a solution: > Per the above, it would not. It would make things worse. This has been > true at every other OSS project I've seen documented (disastrously so > with MySQL); there is no reason to believe that Postgres would be any > different. > > I also do not see why you are so resistant to the idea of documenting a > tracking the post-CF steps so that we can get more people on them. > I love how we all have the same arguments, every year, year after year. So let me just throw in my proverbial two cents. As I see it we can *NOT* increase our development time line. Open Source just doesn't work that way. People want it, and want it now. Period. It will alienate feature contributors and make us fodder for bad press (blogs whatever) when we are lacking in some area where another isn't. We can decrease our development cycle. We could do an Ubuntu (and similarly Fedora) style cycle where people that want the hot new features now, can. They would do this by using our 6 month releases, while stable enterprises would use our LTS release. This is "kind of" happening now with our new Alpha release status. We can release annually and go all balls toward each release. The second option seems to be the middle ground that we will settle on regardless of what arguments are presented. The third option is what I would like to see happen. Which means we would actually have a 9 month development cycle/cutoff and a three month alpha/beta/release. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
>>> Another solution would be to make major releases less frequent. >> That's not a solution and you know it. > > I do? Ok, here's the reasons it's not a solution: 1) having a longer development cycle would frustrate many users who want new features sooner, not later. The current 1 year is a good compromise between reliability and "release often". A longer period would not be. 2) Lengthening the development period would make things less efficient. The amount of effort we need to test, document, integrate, package, etc., gets *greater* per patch when we have hundreds of patches. So if we *planned* an 18-month release, I expect that it would end up being a 24-month release. 3) If we deliberately lengthen the release cycle without doing anything about why the post-CF portion takes so long, it will continue to get longer, indefinitely. Eventually, we're at 3.5 year releases and our users abandoning Postgres for another database who can actually get a release out. 4) It does nothing to address the *contributor* complaint that the non-development part of our dev cycle is too long and keeps getting longer. A longer release cycle would make that worse. If we could concievably do a release every 4 months, I believe that it would be easy to keep the non-development portion of our cycle down to 30% or less. We can't, so we need to look at ways to speed up the work we're already doing. > I have no idea how you know so much about me, but don't realize I was > saying that we should extend the release cycle so we don't release as > often, "make major releases less frequent" (every 12-14 months). This > has nothing to do with how we process the releases, parallel or not. OK, to restate: making the cycle longer will not help the development-to-integration&testing ratio. It will make it worse. > As I have said in the past, we are nearing feature-completeness (in a > way), so having perhaps an 18-month release cycle is an idea. That > would give more time for development compared to beta, etc. Per the above, it would not. It would make things worse. This has been true at every other OSS project I've seen documented (disastrously so with MySQL); there is no reason to believe that Postgres would be any different. I also do not see why you are so resistant to the idea of documenting a tracking the post-CF steps so that we can get more people on them. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Sat, Aug 29, 2009 at 1:05 PM, Bruce Momjian wrote: > Robert Haas wrote: >> Both committers and non-committers are currently suffering from the >> fact that there is not a lot of time in the year which is set aside >> for development, i.e. neither CommitFest-time nor beta-time. To fix >> this problem, we can: >> >> 1. Make CommitFests shorter. >> 2. Make CommitFests less frequent. >> 3. Continue developing during CommitFests. >> 4. Make beta cycles shorter. >> 5. Make beta cycles less frequent (i.e. lengthen the release cycle). >> 6. Continue developing during beta. >> >> I believe (1) to be completely impractical and (3) to be >> self-defeating. I suspect (2) will backfire badly. That doesn't >> leave us with a lot of options. We can certainly do (5), but the >> downside is that features that get committed won't hit release for a >> very long time. I and others have suggested a couple of possible >> approaches toward (4) or (6), such as changing the way we do release >> notes, adding more regression tests to give us more (not perfect) >> confidence that the release is solid, and/or branching the tree during >> beta. None of those ideas have gotten a single vote of confidence >> from you or Bruce. What's your suggestion? > > Another solution would be to make major releases less frequent. I mentioned that one. It's #5. I also mentioned the downside. Downthread you make reference to PostgreSQL being feature-complete, but I don't personally think that's the right way to think about it. The people who are submitting patches are doing so because they feel that the product is missing the features implemented by those patches. Sure, some patches are pure performance plays, or bug fixes, or documentation improvements, but many of them are new features, plain and simple. And the people who submit those patches want to see them in a released version on a reasonable time scale. I do, anyway. I really can't understand why it isn't possible for us to find a way to make an annual release happen, and with more than 8-12 weeks of development time (ie non-CommitFest non-beta) available during that year. I understand that there is a need for some time to be set aside for reviewing, testing, debugging, packaging, and so forth, but the current schedule contemplates that this time is upwards of 75%, and I think that's excessive. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Josh Berkus wrote: > Bruce, > > >> None of those ideas have gotten a single vote of confidence > >> from you or Bruce. What's your suggestion? > > > > Another solution would be to make major releases less frequent. > > That's not a solution and you know it. I do? > Our development cycle has to change with the growth of the project. I > know you don't like change and are comfortable with how we used to do I don't? Wow, you are helping me see the light? > things in 2001. But at this point the old practices are holding us back > and we need to continue growing, or die. > > Our old development cycle was, effectively, single-process just like the > old database engine was once. Making development more efficient and > better for all contributors is largely a process of making it parallel > by the incorporation of more people on every step, which also requires > increased transparency, openness and tracking. > > Otherwise, like an overloaded database application in serializable mode, > our development will just get slower and slower until it stops completely. I have no idea how you know so much about me, but don't realize I was saying that we should extend the release cycle so we don't release as often, "make major releases less frequent" (every 12-14 months). This has nothing to do with how we process the releases, parallel or not. As I have said in the past, we are nearing feature-completeness (in a way), so having perhaps an 18-month release cycle is an idea. That would give more time for development compared to beta, etc. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Bruce, >> None of those ideas have gotten a single vote of confidence >> from you or Bruce. What's your suggestion? > > Another solution would be to make major releases less frequent. That's not a solution and you know it. Our development cycle has to change with the growth of the project. I know you don't like change and are comfortable with how we used to do things in 2001. But at this point the old practices are holding us back and we need to continue growing, or die. Our old development cycle was, effectively, single-process just like the old database engine was once. Making development more efficient and better for all contributors is largely a process of making it parallel by the incorporation of more people on every step, which also requires increased transparency, openness and tracking. Otherwise, like an overloaded database application in serializable mode, our development will just get slower and slower until it stops completely. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Tom Lane wrote: > "Kevin Grittner" writes: > > Robert Haas wrote: > >> The final CommitFest began November 11, 2008. It closed March 25, > >> 2009 (+ 144 days). Beta1 was released April 15, 2009 (+ 21 days). > > > I'm not entirely clear on what was happening during the 21 days > > between the end of the CommitFest and and the release of beta1. > > Release note drafting is one part of it, but mostly it's "loose end > cleanup". Historically there have always been a pile of loose ends > to be dealt with, and the CommitFest structure doesn't really do > anything to avoid that. If you're interested, attached are all the > commits between commitfest closure (which I announced immediately > after committing the addition of contrib/btree_gin) and stamping > beta1 (which was actually several days before the date Robert gives, > because of the need for the packagers to do their thing). > > It appears to me that release notes weren't actually the bottleneck this > time, though they have been in some prior cycles. Bruce committed a > first draft immediately after the commitfest closed. Rather, it was > arguing about things like \df behavior and cardinality() that took up > the time. Yep, the bottom line here is that patches get into CVS, but issues come up related to the patch, and we keep looking for good fixes, but once the final commit-fest is over, we _have_ to fix these issues. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas wrote: > Both committers and non-committers are currently suffering from the > fact that there is not a lot of time in the year which is set aside > for development, i.e. neither CommitFest-time nor beta-time. To fix > this problem, we can: > > 1. Make CommitFests shorter. > 2. Make CommitFests less frequent. > 3. Continue developing during CommitFests. > 4. Make beta cycles shorter. > 5. Make beta cycles less frequent (i.e. lengthen the release cycle). > 6. Continue developing during beta. > > I believe (1) to be completely impractical and (3) to be > self-defeating. I suspect (2) will backfire badly. That doesn't > leave us with a lot of options. We can certainly do (5), but the > downside is that features that get committed won't hit release for a > very long time. I and others have suggested a couple of possible > approaches toward (4) or (6), such as changing the way we do release > notes, adding more regression tests to give us more (not perfect) > confidence that the release is solid, and/or branching the tree during > beta. None of those ideas have gotten a single vote of confidence > from you or Bruce. What's your suggestion? Another solution would be to make major releases less frequent. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Fri, 2009-08-28 at 10:55 -0700, Josh Berkus wrote: > Here is my proposal for CFs for this year: > > We do four CFs, July 15, September 15, November 15, and January 15. > > However, we rigidly apply the 30-day deadline for the January 15 CF. > That is, anything which is not completely ready for commit on February > 14 gets punted to the next version. None of the "oh it's almost ready", > no "just 2 more weeks of review". Make sure everything gets reviewed > promptly (we *can* do it) and commit or punt. Fine for me. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Folks, Here is my proposal for CFs for this year: We do four CFs, July 15, September 15, November 15, and January 15. However, we rigidly apply the 30-day deadline for the January 15 CF. That is, anything which is not completely ready for commit on February 14 gets punted to the next version. None of the "oh it's almost ready", no "just 2 more weeks of review". Make sure everything gets reviewed promptly (we *can* do it) and commit or punt. == Speeding things up == As far as our annual cycle is concerned, I don't think that the development/CF period is the problem; it's as efficient as we really want it to be. It's what comes after: post-CF cleanup, integration, beta. This period is especially a problem because it is one of little visible activity, no development, and generally waiting-room mentality for most of our contributors. Here are my proposals for making all of that go faster, with the caveat that it probably won't get better until the 2nd time we do it: Post-CF: Make a list (now, not in January) of the tasks which need to be done between CFend and Beta. We'll find that some of them could be done by someone other than Tom and Bruce, and that others could be done before CFend. Beta: Create a mailing list (why don't we have a list for testers? is testing less important than the JDBC driver?) and a simple web app or templated wiki page for testers. Allow people to check in with which version they tested (Alpha1,2,3, Beta 1,2,3) and what tests they ran, and issues encountered (if any). We should do this now so we can get started with Alpha testing. When this testing gets into swing, we can also look at recruiting volunteers to run various popular OSS apps' test suites against the developing version of PostgreSQL. Once beta starts, have a list of pending issues in some editable/commentable location (wiki is ok for this, or we can pervert the commitfest app) *as soon as those issues arise* so that as many hackers as possible can work on those issues. We did do a "pending issues" list for 8.4, but not until we were already over a month into beta, and then the list wasn't very accurate. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Ron Mayer wrote: > Josh Berkus wrote: >> There's some very good reasons for the health of the project to >> have specific release dates and stick to them. > > Help me understand why? I don't know how many places are like this, but to get any significant staff or hardware resources officially allocated to anything here, you need a minimum of three months lead time. (Less, of course, if things are crashing and burning around our users' ears; more if the managers don't see an immediate and direct benefit to the users.) Any hope of organized participation by the Wisconsin Courts in a beta program would require a date they can put on their calendars and schedule around with confidence. As it is, what I do is based on having permission to run tests on my own time when there are hardware resources I can find to use which won't disrupt anything. >From my perspective, a hard date for the beta release is more important than a hard date for the production release. Management here is very easy to sell on the concept that PostgreSQL stays in beta testing until there is confidence that the release is stable and trustworthy. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Greg Stark wrote: > They basically don't do any integration testing and leave that up to > the distributions now. Instead they have an "rc" release *every week* > like clockwork and every 2-3 months the last one becomes a new version > regardless of whether there's any notable new feature. They have a two week "merge period" during which all patches of any importance are merged. The RCs are released after that, and they are supposed to include only bugfixes to the merged patches. Some things are still merged after the merge window is closed, but they are very few. They release as many RCs as are necessary to close the majority of bugs/regressions. So yeah, they have some of the "time-based" nature, but they also use the "when it's ready" philosophy a lot. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Fri, Aug 28, 2009 at 7:35 AM, Greg Smith wrote: > It's really amazing that a useful result ever comes out of this model at > all, and I know I'm not alone that I presume all Linux kernel releases are > too full of bugs to be useful until I've proven otherwise with my own QA. > > If the core PostgreSQL development worked like the Linux kernel, at the end > of each CommitFest whatever was done at that point would get published as > the new release. Instead of pausing to focus on a stable release everyone > would just speed ahead, backporting any major issues not discovered until > after the official release. The lesson that they took from earlier releases was that "pausing" development just led to heartache and delays and didn't actually help the release at all. Keep in mind that the Linux kernel itself is just an integration effort now anyways. All the actual development happens in other trees earlier anyways. Patches are only sent up to Linux when they've been fully developed (and hopefully somewhat tested) elsewhere. The arguments against the Linux approach are that a) it involves a lot of backpatching which is a pain. This is less convincing than it appears because the more often you fork branches the less different they all are. b) Our developers are also our testers and we don't have independent distribution vendors available to do testing. Actually we do, aside from people like EDB who have large test suites we're discussing how to get more testing from users precisely because our developers aren't really our testers. The big difference between Linux and ourselves is that it's a lot more work to migrate a database. So nobody would be particularly helped by having frequent releases. It would make a lot of sense even for us when the day comes that most people just run whatever Redhat or Debian ship. In which case we could come out with releases but users would happily ignore those releases until Redhat or Debian picked out, tested it, and released it in their distributions. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 08:02:03PM -0700, Ron Mayer wrote: > Andrew Dunstan wrote: > > I don't know of anyone who is likely to want to try out alphas in their > > normal development environments. The client I approached was > > specifically prepared to test beta releases that way. > > Perhaps end-users won't, but I think companies who develop software that > works on top of postgres will. Perhaps to make sure their existing software > continues to work; or perhaps to get a head start working with new features. > I test against CVS-head occasionally. I've been trying to help a client take up new versions of postgresql more quickly as the performance or feature content is often very valuable to them. Accordingly, I have encouraged them to run periodic samples of the nightly snapshots on at least one development instance, and to run the betas in the test environment. The goal is to be confident on the day of the postgresql release that we have tested enough and fixed any incompatibilities so that, if they so choose, they could migrate production to the new release immediately. This way they get several months extra benefit from improvements to postgresql. -dg -- David Gould da...@sonic.net 510 536 1443510 282 0869 If simplicity worked, the world would be overrun with insects. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Greg Smith wrote: The Linux kernel developers are very clear that they don't care one bit about testing for stability or lack of data loss in any system-oriented way. That's somebody else's job now, typically the Linux distributor who decides which of the kernels floating around are the most stable, hopefully running more and larger tests than the kernel developers do. Right. And we are not in any position to do that, even if we wanted to. The Linux kernel is a great example of how we don't want to do it, IMNSHO. We need to enhance our testing to preserve our reputation for stability and reliability, not throw the responsibility over the fence. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 09:38:15PM +0200, Dimitri Fontaine wrote: > Exactly, and I think that what we're missing here is a simple tool for > our users to check a new PostgreSQL release against their existing > application. > > We already know how to either log all queries and analyze the log files > (CSV makes it easier, pgfouine parses them too) or to have a fe/be > protocol proxy to record application SQL traffic (tsung recorder does > that). > > What we miss is a tool to run the captured queries through both versions > of PG and report any resultset mismatch, of course with a way to account > for ordering issues (but we've seen people rely on the ordering when > they don't give an order by clause, then bug the lists about it if a new > release changes it). This would be very useful. I often am asked "how much better will the new release run our apps" as part of convincing a client to upgrade to a more current postgresql release. Being able to replay a days workload in a somewhat realistic manner would be a great help. -dg -- David Gould da...@sonic.net 510 536 1443510 282 0869 If simplicity worked, the world would be overrun with insects. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, 27 Aug 2009, Ron Mayer wrote: The Linux kernel seems to do fine with a "when it is ready" cycle, where some releases(2.6.24) take twice the time of others(2.6.28)[1,2]. [2] http://fblinux.freebase.com/view/base/fblinux/views/linux_kernel_release That link has bad data. If you check the tagging dates by looking at the source material here: http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.27 http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.28 You can see that 2.6.28 didn't come out until 2008-12-14, not the 2008-10-24 claimed on the freebase.com site. I imagine it has similar stability and lack-of-data-loss requirements as postgres does. The Linux kernel developers are very clear that they don't care one bit about testing for stability or lack of data loss in any system-oriented way. That's somebody else's job now, typically the Linux distributor who decides which of the kernels floating around are the most stable, hopefully running more and larger tests than the kernel developers do. For example, if you consider Ubuntu 9.04 Jaunty, development started with 2.6.27, upgraded to 2.6.28, then rejected moving to 2.6.29 even though they might have slipped it in.[1] When faced with the similar decision for 2.6.26 vs. 2.6.27 in the previous release, they went for the later one, based on the features they needed to be stable.[2] It's really amazing that a useful result ever comes out of this model at all, and I know I'm not alone that I presume all Linux kernel releases are too full of bugs to be useful until I've proven otherwise with my own QA. If the core PostgreSQL development worked like the Linux kernel, at the end of each CommitFest whatever was done at that point would get published as the new release. Instead of pausing to focus on a stable release everyone would just speed ahead, backporting any major issues not discovered until after the official release. [1] http://www.h-online.com/news/No-2-6-29-kernel-for-Jaunty-Jackalope--/112636 [2] https://lists.ubuntu.com/archives/ubuntu-devel/2008-August/026142.html -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Fri, Aug 28, 2009 at 4:39 AM, Ron Mayer wrote: > Josh Berkus wrote: >> There's some very good reasons for the health of the project to have >> specific release dates and stick to them. > > Help me understand why? > > The Linux kernel seems to do fine with a "when it is ready" cycle, > where some releases(2.6.24) take twice the time of others(2.6.28)[1,2]. > I imagine it has similar stability and lack-of-data-loss requirements > as postgres does. Uhm, the Linux release process is actually now the canonical example of time-based release management. They used to do "when it is ready" and that led to such a dramatic catastrophic failure with 2.6 that they adopted the most dogmatically time-based system of any open source project. They basically don't do any integration testing and leave that up to the distributions now. Instead they have an "rc" release *every week* like clockwork and every 2-3 months the last one becomes a new version regardless of whether there's any notable new feature. Other than the first few releases after 2.6.0 when things were still fairly unstable and major fixes were going in the release cycle has been remarkaby regular modulo holidays and vacations: r | d | days --++-- ChangeLog-2.6.30 | 2009-06-10 | 79 ChangeLog-2.6.29 | 2009-03-23 | 89 ChangeLog-2.6.28 | 2008-12-24 | 75 ChangeLog-2.6.27 | 2008-10-10 | 89 ChangeLog-2.6.26 | 2008-07-13 | 87 ChangeLog-2.6.25 | 2008-04-17 | 84 ChangeLog-2.6.24 | 2008-01-24 | 107 ChangeLog-2.6.23 | 2007-10-09 | 92 ChangeLog-2.6.22 | 2007-07-09 | 74 ChangeLog-2.6.21 | 2007-04-26 | 81 ChangeLog-2.6.20 | 2007-02-04 | 67 ChangeLog-2.6.19 | 2006-11-29 | 70 ChangeLog-2.6.18 | 2006-09-20 | 94 ChangeLog-2.6.17 | 2006-06-18 | 90 ChangeLog-2.6.16 | 2006-03-20 | 76 ChangeLog-2.6.15 | 2006-01-03 | 67 ChangeLog-2.6.14 | 2005-10-28 | 60 ChangeLog-2.6.13 | 2005-08-29 | 73 ChangeLog-2.6.12 | 2005-06-17 | 107 ChangeLog-2.6.11 | 2005-03-02 | 68 ChangeLog-2.6.10 | 2004-12-24 | 66 ChangeLog-2.6.9 | 2004-10-19 | 66 ChangeLog-2.6.8 | 2004-08-14 | 59 ChangeLog-2.6.7 | 2004-06-16 | 37 ChangeLog-2.6.6 | 2004-05-10 | 36 ChangeLog-2.6.5 | 2004-04-04 | 24 ChangeLog-2.6.4 | 2004-03-11 | 22 ChangeLog-2.6.3 | 2004-02-18 | 14 ChangeLog-2.6.2 | 2004-02-04 | 26 ChangeLog-2.6.1 | 2004-01-09 | 22 ChangeLog-2.6.0 | 2003-12-18 | (31 rows) -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Josh Berkus wrote: > There's some very good reasons for the health of the project to have > specific release dates and stick to them. Help me understand why? The Linux kernel seems to do fine with a "when it is ready" cycle, where some releases(2.6.24) take twice the time of others(2.6.28)[1,2]. I imagine it has similar stability and lack-of-data-loss requirements as postgres does. I understand why commercial packagers like Ubuntu - especially public ones like Novell and Red Hat who have to forecast earnings - want to schedule their releases. And I can imagine they'd appreciate it if project releases aren't too close to their release schedules (if postgres releases right after they release they suffer from not having the current version; if postgres releases just before, they have limited testing time). [1] http://www.linuxfoundation.org/publications/linuxkerneldevelopment.php [2] http://fblinux.freebase.com/view/base/fblinux/views/linux_kernel_release > So, with that in mind: what is your statement on three versus four > commitfests? Does it make a difference to you? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Andrew Dunstan wrote: > I don't know of anyone who is likely to want to try out alphas in their > normal development environments. The client I approached was > specifically prepared to test beta releases that way. Perhaps end-users won't, but I think companies who develop software that works on top of postgres will. Perhaps to make sure their existing software continues to work; or perhaps to get a head start working with new features. I test against CVS-head occasionally. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 6:09 PM, Tom Lane wrote: > "Kevin Grittner" writes: >> Robert Haas wrote: >>> The final CommitFest began November 11, 2008. It closed March 25, >>> 2009 (+ 144 days). Beta1 was released April 15, 2009 (+ 21 days). > >> I'm not entirely clear on what was happening during the 21 days >> between the end of the CommitFest and and the release of beta1. > > Release note drafting is one part of it, but mostly it's "loose end > cleanup". Historically there have always been a pile of loose ends > to be dealt with, and the CommitFest structure doesn't really do > anything to avoid that. If you're interested, attached are all the > commits between commitfest closure (which I announced immediately > after committing the addition of contrib/btree_gin) and stamping > beta1 (which was actually several days before the date Robert gives, > because of the need for the packagers to do their thing). > > It appears to me that release notes weren't actually the bottleneck this > time, though they have been in some prior cycles. Bruce committed a > first draft immediately after the commitfest closed. Rather, it was > arguing about things like \df behavior and cardinality() that took up > the time. It felt, at the time, like the release notes were holding things up, hence the various and so-far-unsuccesful volunteering related to that task. But it's possible that there was enough parallel activity going on that it wasn't actually so. > We could certainly have released a perhaps-less-polished beta earlier. > I think that the traditional criterion is that we don't release beta1 > as long as there are any known issues that might force an initdb. > We were successful in avoiding a post-beta initdb this time, although > IIRC the majority of release cycles have had one --- so maybe you > could argue that that's not so important. It would certainly be > less important if we had working pg_migrator functionality to ease > the pain of going from beta to final. > > Now that we have the alpha-release mechanism defined, some of the > pressure for a quick beta could be taken off by releasing a final > alpha right after the final commitfest closes. Definitely. Looking at it in hindsight, 3 weeks seems like a reasonable amount of time between the end of the last CommitFest and the start of beta. It felt long at the time, but maybe that's partly because the CommitFest was so intermininable. I think what a lot of this boils down to is that we need a better system for managing the tasks that need to be completed at each stage of the release process, and who is working on each one, and what the status of it is, just as we do for CommitFests. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Simon, > The level of detailed planning happening now is a change for the > community and in general I think it's a good thing. In the past we've > always said it will be shipped when it's ready, and now we seem to be > caught by our own rules. There's no need to make hard decisions now. > Let's keep some flexibility in our thinking. If the structures give us > problems, lets change the structures. The idea is the plans help the > developers, not hinder them or make it harder to include big features. There's some very good reasons for the health of the project to have specific release dates and stick to them. "When will it be released" is an important question to answer, and it's far better for the developers who *aren't* working on big features to not be elastic. So, with that in mind: what is your statement on three versus four commitfests? Does it make a difference to you? -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
David Fetter wrote: How about something in the alphas to the effect of, Using PostgreSQL? Have a development server to spare? Try your application stack on alpha1! We'd love to hear back. Functionality, performance, you name it. I don't know of anyone who is likely to want to try out alphas in their normal development environments. The client I approached was specifically prepared to test beta releases that way. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 03:04:20PM -0400, Robert Haas wrote: > On Thu, Aug 27, 2009 at 2:54 PM, Peter Eisentraut wrote: > > On tor, 2009-08-27 at 09:58 -0400, Robert Haas wrote: > >> To get positive results in which you can have confidence, you > >> have to know that the testing which was done actually did a > >> reasonably good job exercising the code in a way that would have > >> flushed out bugs, had any been present. That sounds a lot like > >> the definition of a regression test suite. Of course, we have > >> that already, but it's nowhere near comprehensive. Maybe we > >> should be looking at an expanded test suite that runs on a time > >> scale of hours rather than seconds. Actually, didn't Peter talk > >> about something like this at PGCon? > > > > Let's look at it this way: If I were writing a compiler, then I > > would have two main test approaches. First, I would have an > > in-tree test suite that compiles a bunch of example code snippets > > and checks that the results are reasonable. Call that a > > regression test. It would be informed by code coverage analysis > > and previously reported bugs. Second, as part of my release > > cycle, I would have an agenda to try to compile a large set of > > real programs against my new compiler version. I would do that > > during the beta period. You will notice that GCC pretty much > > operates that way. > > > > We have regression tests. They could and should be expanded. > > That's a developer job, and we can start working on that now. > > But this discussion was about what to do during beta. And I > > think during beta you want to test PostgreSQL against a large set > > of real applications. But we could try to clarify how to actually > > do that in an organized way. > > > > Now, if you want to improve the regression tests, I would suggest > > going through the commits since 8.4beta and since 8.4.0 final > > release and ask how these problems could have been prevented or > > caught earlier. I suppose a test suite for WAL might be part of > > the answer, but a closer analysis might be insightful. > > What I want to do is address the concern about too much of any given > year being consumed by beta and CommitFest. I'm not sure I know how > to do that though. How about something in the alphas to the effect of, Using PostgreSQL? Have a development server to spare? Try your application stack on alpha1! We'd love to hear back. Functionality, performance, you name it. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
"Kevin Grittner" writes: > Robert Haas wrote: >> The final CommitFest began November 11, 2008. It closed March 25, >> 2009 (+ 144 days). Beta1 was released April 15, 2009 (+ 21 days). > I'm not entirely clear on what was happening during the 21 days > between the end of the CommitFest and and the release of beta1. Release note drafting is one part of it, but mostly it's "loose end cleanup". Historically there have always been a pile of loose ends to be dealt with, and the CommitFest structure doesn't really do anything to avoid that. If you're interested, attached are all the commits between commitfest closure (which I announced immediately after committing the addition of contrib/btree_gin) and stamping beta1 (which was actually several days before the date Robert gives, because of the need for the packagers to do their thing). It appears to me that release notes weren't actually the bottleneck this time, though they have been in some prior cycles. Bruce committed a first draft immediately after the commitfest closed. Rather, it was arguing about things like \df behavior and cardinality() that took up the time. We could certainly have released a perhaps-less-polished beta earlier. I think that the traditional criterion is that we don't release beta1 as long as there are any known issues that might force an initdb. We were successful in avoiding a post-beta initdb this time, although IIRC the majority of release cycles have had one --- so maybe you could argue that that's not so important. It would certainly be less important if we had working pg_migrator functionality to ease the pain of going from beta to final. Now that we have the alpha-release mechanism defined, some of the pressure for a quick beta could be taken off by releasing a final alpha right after the final commitfest closes. regards, tom lane 2009-04-09 20:20 scrappy * configure, configure.in, doc/bug.template, src/include/pg_config.h.win32: commit and tag beta1 2009-04-09 19:22 tgl * doc/src/sgml/release.sgml: Update release notes through yesterday; some minor wordsmithing. 2009-04-09 18:32 momjian * doc/src/sgml/monitoring.sgml: Clarify documentation references to pg_stat_get_blocks_fetched and pg_stat_get_blocks_hit, per suggestion from Robert Haas. 2009-04-09 17:50 momjian * src/tools/RELEASE_CHANGES: No more need to update FAQs. 2009-04-09 17:35 petere * src/tools/RELEASE_CHANGES: Add URL for config.guess/sub updates 2009-04-09 17:33 petere * config/: config.guess, config.sub: Update config.guess and config.sub 2009-04-09 16:50 tgl * src/timezone/data/: africa, asia, leapseconds, northamerica, southamerica (REL8_1_STABLE), africa, asia, leapseconds, northamerica, southamerica (REL8_3_STABLE), africa, asia, leapseconds, northamerica, southamerica (REL8_0_STABLE), africa, asia, leapseconds, northamerica, southamerica (REL8_2_STABLE), africa, asia, leapseconds, northamerica, southamerica: Update time zone data files to tzdata release 2009e: DST law changes in Argentina/San_Luis, Cuba, Jordan (historical correction only), Morocco, Palestine, Syria, Tunisia. 2009-04-09 15:38 petere * src/: backend/nls.mk, backend/po/de.po, backend/po/es.po, backend/po/fr.po, backend/po/ja.po, backend/po/nl.po, backend/po/ru.po, backend/po/tr.po, bin/initdb/nls.mk, bin/initdb/po/de.po, bin/initdb/po/es.po, bin/initdb/po/fr.po, bin/initdb/po/ja.po, bin/initdb/po/ru.po, bin/pg_config/nls.mk, bin/pg_config/po/de.po, bin/pg_config/po/fr.po, bin/pg_config/po/ja.po, bin/pg_config/po/ru.po, bin/pg_config/po/tr.po, bin/pg_controldata/nls.mk, bin/pg_controldata/po/de.po, bin/pg_controldata/po/fr.po, bin/pg_controldata/po/ja.po, bin/pg_ctl/nls.mk, bin/pg_ctl/po/cs.po, bin/pg_ctl/po/de.po, bin/pg_ctl/po/fr.po, bin/pg_ctl/po/ja.po, bin/pg_ctl/po/ru.po, bin/pg_dump/nls.mk, bin/pg_dump/po/de.po, bin/pg_dump/po/fr.po, bin/pg_dump/po/ja.po, bin/pg_dump/po/tr.po, bin/pg_resetxlog/nls.mk, bin/pg_resetxlog/po/de.po, bin/pg_resetxlog/po/fr.po, bin/pg_resetxlog/po/ja.po, bin/pg_resetxlog/po/ru.po, bin/psql/nls.mk, bin/psql/po/de.po, bin/psql/po/es.po, bin/psql/po/fr.po, bin/psql/po/ja.po, bin/psql/po/tr.po, bin/scripts/nls.mk, bin/scripts/po/de.po, bin/scripts/po/fr.po, bin/scripts/po/ja.po, interfaces/ecpg/ecpglib/nls.mk, interfaces/ecpg/ecpglib/po/de.po, interfaces/ecpg/ecpglib/po/es.po, interfaces/ecpg/ecpglib/po/fr.po, interfaces/ecpg/preproc/nls.mk, interfaces/ecpg/preproc/po/de.po, interfaces/ecpg/preproc/po/es.po, interfaces/ecpg/preproc/po/fr.po, interfaces/libpq/nls.mk, interfaces/libpq/po/de.po, interfaces/libpq/po/fr.po, interfaces/li
Re: [HACKERS] 8.5 release timetable, again
Kevin Grittner escribió: > Robert Haas wrote: > > > The final CommitFest began November 11, 2008. It closed March 25, > > 2009 (+ 144 days). Beta1 was released April 15, 2009 (+ 21 days). > > I'm not entirely clear on what was happening during the 21 days > between the end of the CommitFest and and the release of beta1. I > seem to remember Bruce saying that there were bugs being fixed, and > that it didn't make sense to release a beta with known bugs of that > magnitude, but I'm not clear on what was up with that. That's nonsense IMHO. We should have just released a version documenting the known bugs, and asking for people to test the rest of the system. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas escribió: > Of course I don't think we'd actually need to start a CommitFest quite > as quickly as we did this time, because with a shorter release cycle > there ought to be a lot less patches already accumulated by the time > we release, especially if there are clearly defined tasks for > developers to do during the beta period. On the other hand, 8.4beta > was arguably too short, since we missed some serious problems, so the > picture above may be a bit too rosy. Maybe what this says is that we need to get a "pre-beta" release out as early as possible, just after finalizing the last commitfest, and before the "open items" and Bruce-approved release note writing are sorted out. Probably the last alpha release will fill that role. Sadly, the greek folk did not consider having a letter between alpha and beta :-( -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas wrote: > The final CommitFest began November 11, 2008. It closed March 25, > 2009 (+ 144 days). Beta1 was released April 15, 2009 (+ 21 days). I'm not entirely clear on what was happening during the 21 days between the end of the CommitFest and and the release of beta1. I seem to remember Bruce saying that there were bugs being fixed, and that it didn't make sense to release a beta with known bugs of that magnitude, but I'm not clear on what was up with that. Did we close the CF with known bugs open, or were these missed in the CF and found after? Surely it shouldn't normally take three weeks to get a beta test version to the public after we close the last CF? Just looking for where we could pick up a few weeks more of development in each year -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 04:22:58PM -0400, Jonah H. Harris wrote: > On Thu, Aug 27, 2009 at 3:53 PM, David Fetter wrote: > > > > I would appreciate it if somebody could send out some messages > > > of calm, while I/we work. The time for open review will come > > > around soon enough. > > > > With all due respect, the time for open review is now. You have > > already tried closed development several times, and it each time > > has been, more or less, a spectacular failure. > > Unlike Robert and Heikki, I don't see you contributing to or > assisting Simon's work. My assistance is of the form of actually getting it done. Simon's work is absolutely fantastic, but only when the rest of the community can actually help. When it shows up late, it actually hurts everybody, most of all Simon. > And, while I may be wrong, I doubt that you assisted in funding any > of Simon's work on hot standby either. As such, it's my opinion > that continuing to criticize him from the sidelines is not only > rude, but is also a bad idea as it relates to his motivation in > working on this feature. If, "your past strategy has a track record of failure, go with a new strategy, one pretty much universally adopted in PostgreSQL," will demotivate someone, I can't help that, and I doubt it's actually true. I'm trying to help here, and encouraging a failed strategy is not helping. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
and...@dunslane.net (Andrew Dunstan) writes: > Actually, what I had in mind was getting people to run their > applications etc. in some non-production environment on the beta. I > talked to a client today and he said "sure, we have several > development environments and we can put one or two on the beta and > then let the developers just do their thing on it." Testing the things > we know about is in a way less important than making sure nothing else > got broken. I've gotten the DB work on one of our applications to the point where there's a meaningful set of DB tests that can be run in automated fashion. As a result, I rotate between PG versions periodically; every couple weeks, I recompile HEAD, and run a test against it to make sure I don't see any regressions. It would be insane to think about deploying on some 8.5-alpha version, but it's nice to have something I can run in ~5 minutes that exercises a fair bit of at least vaguely realistic functionality. -- "cbbrowne","@","ca.afilias.info" Christopher Browne "Bother," said Pooh, "Eeyore, ready two photon torpedoes and lock phasers on the Heffalump, Piglet, meet me in transporter room three" -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 3:56 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> What I want to do is address the concern about too much of any given >> year being consumed by beta and CommitFest. I'm not sure I know how >> to do that though. > > How much time were we in beta? I thought most time was spent trying to > get to beta in the first place. [ looks ] The final CommitFest began November 11, 2008. It closed March 25, 2009 (+ 144 days). Beta1 was released April 15, 2009 (+ 21 days). 8.4.0 was released July 1, 2009 (+ 77 days). The first CommitFest for 8.5 began on July 15, 2009 (+ 14 days). http://www.postgresql.org/about/news.1074 http://wiki.postgresql.org/index.php?title=CommitFest_2008-11&action=history http://www.postgresql.org/docs/8.4/static/release-8-4.html In total, the tree was closed for 256 days, or 8.5 months, of which the final CommitFest accounted for approximately 56%. Had we closed the final CommitFest in 30 days rather than 144 days, and had everything else taken the same amount of time, the release would have occurred on March 9th and the first CommitFest for 8.5 would have started on March 23rd. Hmm... maybe that's not actually that bad. If we stuck to a similar schedule for 8.5, but with a timely last CF, then we'd have either (3 CF): 2009-11-15 Final CommitFest Begins 2009-12-15 Final CommitFest Ends 2010-01-05 Beta 2010-03-23 Release 2010-04-06 First CommitFest for 8.6 Begins Or (4 CF): 2010-01-15 Final CommitFest Begins 2010-02-15 Final CommitFest Ends 2010-03-08 Beta 2010-05-24 Release 2010-06-07 First CommitFest for 8.6 Begins Of course I don't think we'd actually need to start a CommitFest quite as quickly as we did this time, because with a shorter release cycle there ought to be a lot less patches already accumulated by the time we release, especially if there are clearly defined tasks for developers to do during the beta period. On the other hand, 8.4beta was arguably too short, since we missed some serious problems, so the picture above may be a bit too rosy. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 3:53 PM, David Fetter wrote: > > I would appreciate it if somebody could send out some messages of > > calm, while I/we work. The time for open review will come around > > soon enough. > > With all due respect, the time for open review is now. You have > already tried closed development several times, and it each time has > been, more or less, a spectacular failure. Unlike Robert and Heikki, I don't see you contributing to or assisting Simon's work. And, while I may be wrong, I doubt that you assisted in funding any of Simon's work on hot standby either. As such, it's my opinion that continuing to criticize him from the sidelines is not only rude, but is also a bad idea as it relates to his motivation in working on this feature. -- Jonah H. Harris, Senior DBA myYearbook.com
Re: [HACKERS] 8.5 release timetable, again
Robert Haas escribió: > What I want to do is address the concern about too much of any given > year being consumed by beta and CommitFest. I'm not sure I know how > to do that though. How much time were we in beta? I thought most time was spent trying to get to beta in the first place. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 08:48:43PM +0100, Simon Riggs wrote: > > On Sun, 2009-08-23 at 01:57 -0400, Tom Lane wrote: > > Robert Haas writes: > > > I posted a note about a week ago which drew far less commentary > > > than I expected, regarding the timetable for the release of 8.5. > > > > > http://archives.postgresql.org/pgsql-hackers/2009-08/msg01256.php > > > > > Perhaps this is already being discussed on -core, but if so the > > > conclusions haven't been shared publicly. > > > > Core hasn't discussed 8.5 schedule since the discussions that > > Peter summarized in the message you cited above. I share your > > concern that "release in time for PGCon" isn't very realistic if > > we don't get more aggressive about schedule. On the other hand, > > we didn't get all that much done in this fest. If we cut back to > > a three-fest schedule I think we may succeed in releasing 8.5 > > early, but only because there is nothing interesting in it :-(. > > Dunno where the right balance is. > > General comment on thread: > > The level of detailed planning happening now is a change for the > community and in general I think it's a good thing. In the past > we've always said it will be shipped when it's ready, and now we > seem to be caught by our own rules. There's no need to make hard > decisions now. Let's keep some flexibility in our thinking. If the > structures give us problems, lets change the structures. The idea is > the plans help the developers, not hinder them or make it harder to > include big features. > > In my view it is important that I have a holistic view of what I am > doing and that means it is very difficult for others to help in a > way that doesn't merely hinder me (or Fujii-san). Speed of coding is > not the issue here, nor is the number of hands on a keyboard. > > I don't share the doubts or fears expressed that the two big patches > will not make it into Postgres in this release. We now have more > people experienced in these areas of code than at any other time in > our history. We have almost-complete solutions from experienced > developers. In particular, I have faith in Fujii-san. > > I would appreciate it if somebody could send out some messages of > calm, while I/we work. The time for open review will come around > soon enough. With all due respect, the time for open review is now. You have already tried closed development several times, and it each time has been, more or less, a spectacular failure. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Sun, 2009-08-23 at 01:57 -0400, Tom Lane wrote: > Robert Haas writes: > > I posted a note about a week ago which drew far less commentary than I > > expected, regarding the timetable for the release of 8.5. > > > http://archives.postgresql.org/pgsql-hackers/2009-08/msg01256.php > > > Perhaps this is already being discussed on -core, but if so the > > conclusions haven't been shared publicly. > > Core hasn't discussed 8.5 schedule since the discussions that Peter > summarized in the message you cited above. I share your concern that > "release in time for PGCon" isn't very realistic if we don't get more > aggressive about schedule. On the other hand, we didn't get all that > much done in this fest. If we cut back to a three-fest schedule > I think we may succeed in releasing 8.5 early, but only because there > is nothing interesting in it :-(. Dunno where the right balance is. General comment on thread: The level of detailed planning happening now is a change for the community and in general I think it's a good thing. In the past we've always said it will be shipped when it's ready, and now we seem to be caught by our own rules. There's no need to make hard decisions now. Let's keep some flexibility in our thinking. If the structures give us problems, lets change the structures. The idea is the plans help the developers, not hinder them or make it harder to include big features. In my view it is important that I have a holistic view of what I am doing and that means it is very difficult for others to help in a way that doesn't merely hinder me (or Fujii-san). Speed of coding is not the issue here, nor is the number of hands on a keyboard. I don't share the doubts or fears expressed that the two big patches will not make it into Postgres in this release. We now have more people experienced in these areas of code than at any other time in our history. We have almost-complete solutions from experienced developers. In particular, I have faith in Fujii-san. I would appreciate it if somebody could send out some messages of calm, while I/we work. The time for open review will come around soon enough. Faith and patience. Please. No need for Fawlty Towers re-runs. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Hi, Robert Haas writes: > On Thu, Aug 27, 2009 at 2:54 PM, Peter Eisentraut wrote: >> We have regression tests. They could and should be expanded. That's a >> developer job, and we can start working on that now. But this >> discussion was about what to do during beta. And I think during beta >> you want to test PostgreSQL against a large set of real applications. >> But we could try to clarify how to actually do that in an organized >> way. Exactly, and I think that what we're missing here is a simple tool for our users to check a new PostgreSQL release against their existing application. We already know how to either log all queries and analyze the log files (CSV makes it easier, pgfouine parses them too) or to have a fe/be protocol proxy to record application SQL traffic (tsung recorder does that). What we miss is a tool to run the captured queries through both versions of PG and report any resultset mismatch, of course with a way to account for ordering issues (but we've seen people rely on the ordering when they don't give an order by clause, then bug the lists about it if a new release changes it). > What I want to do is address the concern about too much of any given > year being consumed by beta and CommitFest. I'm not sure I know how > to do that though. To do this I think we *need* to provide beta tester a validator kind of tool which they can use even without having an application unit test suite. And a way to easily report success or failure, and in case of success, a notion of their tests coverage (which is reduced to the list of PG features the application exercices, but an automated way to extract the information while running the tool would allow for hackers friendly categories). Anyone interrested into starting a project and coding the tool we so much need? Regards, -- dim PS: of course provided with such a tool, I'd run it for several applications as soon as possible, which might include every alpha release. PPS: I'll be happy to participate into such a project, but I seem to have a rather full plate already and a shrinking OpenSource free time, so waiting for me to get there won't help until it's about too late. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 2:54 PM, Peter Eisentraut wrote: > On tor, 2009-08-27 at 09:58 -0400, Robert Haas wrote: >> To get positive results in which you can have confidence, you have to >> know that the testing which was done actually did a reasonably good >> job exercising the code in a way that would have flushed out bugs, had >> any been present. That sounds a lot like the definition of a >> regression test suite. Of course, we have that already, but it's >> nowhere near comprehensive. Maybe we should be looking at an expanded >> test suite that runs on a time scale of hours rather than seconds. >> Actually, didn't Peter talk about something like this at PGCon? > > Let's look at it this way: If I were writing a compiler, then I would > have two main test approaches. First, I would have an in-tree test > suite that compiles a bunch of example code snippets and checks that the > results are reasonable. Call that a regression test. It would be > informed by code coverage analysis and previously reported bugs. > Second, as part of my release cycle, I would have an agenda to try to > compile a large set of real programs against my new compiler version. > I would do that during the beta period. You will notice that GCC pretty > much operates that way. > > We have regression tests. They could and should be expanded. That's a > developer job, and we can start working on that now. But this > discussion was about what to do during beta. And I think during beta > you want to test PostgreSQL against a large set of real applications. > But we could try to clarify how to actually do that in an organized way. > > Now, if you want to improve the regression tests, I would suggest going > through the commits since 8.4beta and since 8.4.0 final release and ask > how these problems could have been prevented or caught earlier. I > suppose a test suite for WAL might be part of the answer, but a closer > analysis might be insightful. What I want to do is address the concern about too much of any given year being consumed by beta and CommitFest. I'm not sure I know how to do that though. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On tor, 2009-08-27 at 09:58 -0400, Robert Haas wrote: > To get positive results in which you can have confidence, you have to > know that the testing which was done actually did a reasonably good > job exercising the code in a way that would have flushed out bugs, had > any been present. That sounds a lot like the definition of a > regression test suite. Of course, we have that already, but it's > nowhere near comprehensive. Maybe we should be looking at an expanded > test suite that runs on a time scale of hours rather than seconds. > Actually, didn't Peter talk about something like this at PGCon? Let's look at it this way: If I were writing a compiler, then I would have two main test approaches. First, I would have an in-tree test suite that compiles a bunch of example code snippets and checks that the results are reasonable. Call that a regression test. It would be informed by code coverage analysis and previously reported bugs. Second, as part of my release cycle, I would have an agenda to try to compile a large set of real programs against my new compiler version. I would do that during the beta period. You will notice that GCC pretty much operates that way. We have regression tests. They could and should be expanded. That's a developer job, and we can start working on that now. But this discussion was about what to do during beta. And I think during beta you want to test PostgreSQL against a large set of real applications. But we could try to clarify how to actually do that in an organized way. Now, if you want to improve the regression tests, I would suggest going through the commits since 8.4beta and since 8.4.0 final release and ask how these problems could have been prevented or caught earlier. I suppose a test suite for WAL might be part of the answer, but a closer analysis might be insightful. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Greg Stark writes: > On Thu, Aug 27, 2009 at 3:11 PM, Tom Lane wrote: >> ... However this is quite haphazard since (a) the regression tests >> aren't especially designed to exercise all of the WAL logic, and (b) >> pg_dump might not show the effects of some problems, particularly not >> corruption in non-system indexes. It would be worth the trouble to >> create a more specific test methodology. > What I've been thinking of doing is having the regression suite take a > backup after initdb and set archive mode on. when the regression suite > finishes start the backup up and replay all the WAL. > I'm not sure how to compare the databases since I don't think pg_dump > actually works here -- a lot of the data is dropped by the end of the > test. Yeah, that's another problem with using the existing tests for this purpose --- a lot of possibly-useful stuff isn't kept around to the end. And the desire to keep the test modules independent limits the amount of interaction between them too. I really think we'd need a bespoke set of tests to get very far with this. This reminds me that pg_dump/pg_restore is another large pile of code that receives no formalized testing whatsoever ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 10:38 AM, Greg Stark wrote: > > What I've been thinking of doing is having the regression suite take a > backup after initdb and set archive mode on. when the regression suite > finishes start the backup up and replay all the WAL. > > I'm not sure how to compare the databases - execute 60 of the 121 tests (or at least those that create tables and insert/update/delete the most data) - crash the server and replay the WAL - execute the rest of the tests and cross your fingers :) -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 11:29 AM, Tom Lane wrote: > Robert Haas writes: >> Well, I wasn't suggesting adding a lot more testing of things that >> we're already testing. I was assuming that we would craft the >> additional tests to hit areas that we are not now covering well. My >> point here is only to what Peter said upthread: we want to be able to >> get positive results rather than waiting for "enough" negative results >> (whatever that means). To get positive results, you must have a test >> suite. While letting beta testers test whatever they want has some >> value, testing things we think might be likely hiding places for bugs >> (such as WAL recovery) has merit, too. Making those tests >> well-organized and easily repeatable is, IMHO, a Good Thing. > > The problem here is the "easily repeatable" bit. Almost by definition, > easily repeatable tests don't find hard-to-reproduce problems. I don't > mean to suggest that they're without value, but they are no substitute > for beta testers doing unpredictable things. I think you're overstating the case, but I don't want to argue the point, particularly. What I do want to do is find a way to address the problem described in the last sentence of this email: http://archives.postgresql.org/pgsql-hackers/2009-08/msg01614.php Both committers and non-committers are currently suffering from the fact that there is not a lot of time in the year which is set aside for development, i.e. neither CommitFest-time nor beta-time. To fix this problem, we can: 1. Make CommitFests shorter. 2. Make CommitFests less frequent. 3. Continue developing during CommitFests. 4. Make beta cycles shorter. 5. Make beta cycles less frequent (i.e. lengthen the release cycle). 6. Continue developing during beta. I believe (1) to be completely impractical and (3) to be self-defeating. I suspect (2) will backfire badly. That doesn't leave us with a lot of options. We can certainly do (5), but the downside is that features that get committed won't hit release for a very long time. I and others have suggested a couple of possible approaches toward (4) or (6), such as changing the way we do release notes, adding more regression tests to give us more (not perfect) confidence that the release is solid, and/or branching the tree during beta. None of those ideas have gotten a single vote of confidence from you or Bruce. What's your suggestion? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 11:29:42AM -0400, Tom Lane wrote: > Robert Haas writes: > > Well, I wasn't suggesting adding a lot more testing of things that > > we're already testing. I was assuming that we would craft the > > additional tests to hit areas that we are not now covering well. My > > point here is only to what Peter said upthread: we want to be able to > > get positive results rather than waiting for "enough" negative results > > (whatever that means). To get positive results, you must have a test > > suite. While letting beta testers test whatever they want has some > > value, testing things we think might be likely hiding places for bugs > > (such as WAL recovery) has merit, too. Making those tests > > well-organized and easily repeatable is, IMHO, a Good Thing. > > The problem here is the "easily repeatable" bit. Almost by definition, > easily repeatable tests don't find hard-to-reproduce problems. I don't > mean to suggest that they're without value, but they are no substitute > for beta testers doing unpredictable things. I've wondered before about using a system emulator to snapshot the disk on each write (I'd expect you could put some pretty low level hooks in with qemu and gdb) and then run each snapshot in another system to make sure that either the transaction is rolled back or committed as appropriate. I guess this would take a while to run but may help catch some obscure bugs. Or this an area that's well tested already? -- Sam http://samason.me.uk/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 3:11 PM, Tom Lane wrote: > > Precisely... > > What I'd like to see is some sort of test mechanism for WAL recovery. > What I've done sometimes in the past (and recently had to fix the tests > to re-enable) is to kill -9 a backend immediately after running the > regression tests, let the system replay the WAL for the tests, and then > take a pg_dump and compare that to the dump gotten after a conventional > run. However this is quite haphazard since (a) the regression tests > aren't especially designed to exercise all of the WAL logic, and (b) > pg_dump might not show the effects of some problems, particularly not > corruption in non-system indexes. It would be worth the trouble to > create a more specific test methodology. What I've been thinking of doing is having the regression suite take a backup after initdb and set archive mode on. when the regression suite finishes start the backup up and replay all the WAL. I'm not sure how to compare the databases since I don't think pg_dump actually works here -- a lot of the data is dropped by the end of the test. But at least that would test that replaying WAL didn't cause any crashes. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas writes: > Well, I wasn't suggesting adding a lot more testing of things that > we're already testing. I was assuming that we would craft the > additional tests to hit areas that we are not now covering well. My > point here is only to what Peter said upthread: we want to be able to > get positive results rather than waiting for "enough" negative results > (whatever that means). To get positive results, you must have a test > suite. While letting beta testers test whatever they want has some > value, testing things we think might be likely hiding places for bugs > (such as WAL recovery) has merit, too. Making those tests > well-organized and easily repeatable is, IMHO, a Good Thing. The problem here is the "easily repeatable" bit. Almost by definition, easily repeatable tests don't find hard-to-reproduce problems. I don't mean to suggest that they're without value, but they are no substitute for beta testers doing unpredictable things. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 10:11 AM, Tom Lane wrote: > Robert Haas writes: >> ... That sounds a lot like the definition of a >> regression test suite. Of course, we have that already, but it's >> nowhere near comprehensive. Maybe we should be looking at an expanded >> test suite that runs on a time scale of hours rather than seconds. > > mysql's got one of those, and I haven't noticed that it's kept their > defect rate down any. Hour-long regression suites are the sort of > thing developers won't run. Well, I'll run them. And I bet we could get volunteers to provide machines to run them every night, too, against CVS HEAD. This has been discussed before, and I wasn't the one who suggested it. I can't speak to the value (or lack thereof) of mysql's regression test suite as I know nothing about it, but even if it is completely worthless that does not prove that a worthwhile test suite can't be constructed. > Worse, regression suites are necessarily > designed to exercise only 100%-reproducible behavior. That is true, but our current testing methodology (hoping the beta-testers find the bugs) seems not to be completely satisfactory either. Which brings us to: >> I don't think that any test suite is going to eliminate the need for >> beta-testing. > > Precisely... > > What I'd like to see is some sort of test mechanism for WAL recovery. > What I've done sometimes in the past (and recently had to fix the tests > to re-enable) is to kill -9 a backend immediately after running the > regression tests, let the system replay the WAL for the tests, and then > take a pg_dump and compare that to the dump gotten after a conventional > run. However this is quite haphazard since (a) the regression tests > aren't especially designed to exercise all of the WAL logic, and (b) > pg_dump might not show the effects of some problems, particularly not > corruption in non-system indexes. It would be worth the trouble to > create a more specific test methodology. Yep. I was thinking about this as an area for possible improvement. I don't immediately have a brilliant idea how to do it. I did have the idea of creating a loadable C function which would panic the database. This could be used to crash the database at a particular point (even in mid-query, with sufficient creativity). I think we would certainly need some more powerful way of checking pass/failure than exact-text comparisons on SQL query results. Being a perl guy, the first thing that occurs to me is to write some kind of test harness in perl that can issue SQL queries as well as do other things, but I don't have an exact design mapped out in my head, and I'm sure there are other viable approaches. > In short: merely making the tests bigger doesn't impress me in the > least. Focused testing on areas we aren't covering at all could be > worth the trouble. Well, I wasn't suggesting adding a lot more testing of things that we're already testing. I was assuming that we would craft the additional tests to hit areas that we are not now covering well. My point here is only to what Peter said upthread: we want to be able to get positive results rather than waiting for "enough" negative results (whatever that means). To get positive results, you must have a test suite. While letting beta testers test whatever they want has some value, testing things we think might be likely hiding places for bugs (such as WAL recovery) has merit, too. Making those tests well-organized and easily repeatable is, IMHO, a Good Thing. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas writes: > ... That sounds a lot like the definition of a > regression test suite. Of course, we have that already, but it's > nowhere near comprehensive. Maybe we should be looking at an expanded > test suite that runs on a time scale of hours rather than seconds. mysql's got one of those, and I haven't noticed that it's kept their defect rate down any. Hour-long regression suites are the sort of thing developers won't run. Worse, regression suites are necessarily designed to exercise only 100%-reproducible behavior. > I don't think that any test suite is going to eliminate the need for > beta-testing. Precisely... What I'd like to see is some sort of test mechanism for WAL recovery. What I've done sometimes in the past (and recently had to fix the tests to re-enable) is to kill -9 a backend immediately after running the regression tests, let the system replay the WAL for the tests, and then take a pg_dump and compare that to the dump gotten after a conventional run. However this is quite haphazard since (a) the regression tests aren't especially designed to exercise all of the WAL logic, and (b) pg_dump might not show the effects of some problems, particularly not corruption in non-system indexes. It would be worth the trouble to create a more specific test methodology. In short: merely making the tests bigger doesn't impress me in the least. Focused testing on areas we aren't covering at all could be worth the trouble. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas wrote: > Maybe we should be looking at an expanded test suite that runs on a > time scale of hours rather than seconds. > if we could say that we had a regression test suite which covered X% > of our code, and it passed on all Y platforms tested, that would > certainly be a confidence booster, especially for large values of X. > Part of the question, of course, is how to build up such a > regression test suite. Aren't there code coverage monitoring tools that could be run during regression tests? Sure it would take some time to review the results and fashion tests to exercise chunks of code which were missed, but at least we could quantify X and try to make incremental progress on increasing it -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Wed, Aug 26, 2009 at 7:39 PM, Bruce Momjian wrote: > Peter Eisentraut wrote: >> Much of the delay and uncertainty during beta in my mind comes from the >> situation that we wait for negative results and don't trust the release >> until we have seen and fixed enough of them. Instead of waiting for >> concrete, positive results and producing the release with confidence. > > Yep, that is our dilemma. To get positive results in which you can have confidence, you have to know that the testing which was done actually did a reasonably good job exercising the code in a way that would have flushed out bugs, had any been present. That sounds a lot like the definition of a regression test suite. Of course, we have that already, but it's nowhere near comprehensive. Maybe we should be looking at an expanded test suite that runs on a time scale of hours rather than seconds. Actually, didn't Peter talk about something like this at PGCon? I don't think that any test suite is going to eliminate the need for beta-testing. But if we could say that we had a regression test suite which covered X% of our code, and it passed on all Y platforms tested, that would certainly be a confidence booster, especially for large values of X. What we're basically doing now is hoping that beta-testers come up with some novel tests, and that's a bit hit-or-miss. Part of the question, of course, is how to build up such a regression test suite. One way to do it is try to add a test every time you fix a bug, such that the new test fails on the unpatched code and passes with the fix... or we could have a expanded-regression-test only commitfest during beta, or something. I think people would be willing to put in some work on this. I certainly would. I run the regression tests frequently during development but unless I'm making system catalog changes they rarely catch anything. I'm not prepared to write the whole thing myself, but I would definitely be willing to develop and contribute some tests. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 2:30 AM, Stephen Frost wrote: > I agree entirely with Andrew here- what we need are a set of users who > would be willing to run their actual applications against a beta release > in a testing environment. The Beta-Mom position would be working with > some list of users who've volunteered to do that; prodding them when a > new beta comes out, poking them for feedback, working with them on > issues they run into, etc. I don't think we need the Beta-Mom to work with them on issues they run into. We want to hear those things on the mailing lists. But I do think the idea of having someone being Beta-Mom is a good idea because of this simple point: right now we don't know if we have any beta testers. All we hear about is when they run into problems. What we want to know is how many people have run their applications and *not* had problems. So I see the Beta-Mom as being in charge of: Gathering a list of volunteers willing to report results For each volunteer getting a list of features they tested For each volunteer getting a "yay" or "nay" report Ideally we would want to know that people have run their application frameworks for a certain length of time, with slave databases set up, with slony set up, tested failover, with various pluggable langauges, with various contrib modules, with large objects, with full text search, etc. Once we have a reasonable number of positive reports covering a wide range of heavy-duty configuration then we can start to base our confidence with going ahead with a release on those positive reports rather than simply on the amount of time that's passed without a problem. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Tom Lane wrote: > Josh Berkus writes: >>> That is a slightly alarmist. Who are we going to lose these users to? > >> Drizzle. MySQL forks. CouchDB. Any database which has replication >> which you don't need a professional DBA to understand. Whether or not >> it works. > > You haven't explained why we'd lose such folk next year when we haven't > lost them already. MySQL has had replication (or at least has checked > off the bullet point ;-)) for years. I think it's a slow but ongoing stream of organizations that are switching away using logic similar to the thoughts outlined here: http://archives.postgresql.org/pgsql-hackers/2008-05/msg00955.php "...switched their bugzilla from Postgres to MySQL because the admins didn't want to deal with Slony any more. People want simple." MySQL may not have caught postgres in a number of ways yet, but it's good enough now for many of the things it wasn't good enough for earlier. And if it's good enough and easier, it's easy to switch. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
* Andrew Dunstan (and...@dunslane.net) wrote: > Actually, what I had in mind was getting people to run their > applications etc. in some non-production environment on the beta. I > talked to a client today and he said "sure, we have several development > environments and we can put one or two on the beta and then let the > developers just do their thing on it." Testing the things we know about > is in a way less important than making sure nothing else got broken. I agree entirely with Andrew here- what we need are a set of users who would be willing to run their actual applications against a beta release in a testing environment. The Beta-Mom position would be working with some list of users who've volunteered to do that; prodding them when a new beta comes out, poking them for feedback, working with them on issues they run into, etc. The other possible group of users are those who are interested and willing to beta-test actual new external-facing features. That'd be great to have as well, but I don't believe is as important. New features having bugs are a much smaller impact, overall, than bugs which have been introduced in existing code-paths due to changes. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] 8.5 release timetable, again
On Aug 26, 2009, at 8:17 AM, Jean-Michel Pouré wrote: Le mercredi 26 août 2009 à 01:36 -0600, Rick Gigger a écrit : One possible reason that replication is more critical now than it was a year ago is the rise in cloud computing. The ability to fire up instances on demand is much more useful when some of those boxes can be database servers and those databases servers can replicate the primary database and start doing something useful. As far as I can tell this one feature alone is the one thing that makes it hard to convince people to migrate away from Mysql despite it's demonstrable inferiority in many other areas. I think you should have a deep look at these two manuals that I wrote for Drupal: Guidelines for writing MySQL and PostgreSQL compliant SQL http://drupal.org/node/14 and Quidelines for writing SQL efficient code: http://drupal.org/node/559302 I have been using PostgreSQL since 1998. I took part in the development of pgAdmin 1 and pgAdmin 2. I am very proud of the PostgreSQL community, but I think it should fix some important issues in the domain of SQL compliance and compatibility. When reading developers comments on Drupal web site, it seems that there is deep misunderstanding between developers and SQL databases. I would say that 1% of developers know database technology. For example, most Drupal developers think that an INNER JOIN is faster than a LEFT JOIN. The reality of facts is that developers will not even test PostgreSQL and stop using it after the first SQL warning or error. So I would recommend focussing on usability. Then you can work on replication and materilized views. You probably know that a cloud requires several computers. With materialized view, a single computer can perform 100 times better. I see plenty of of possibilities to improve speed using materialized views. But first and firstly ... focus on usability. Then make a pre- release of a PostgreSQL 8.4.2 release or so ... We need to wipe out this MySQL issue once for all. If there is a compat MySQL mode or functions, then include it in core. This is too important for PostgreSQL success. Why MySQL usability is achieved add materialized views and MySQL is dead in the performance statistics, beaten 99% by PostgreSQL. This may be your experience and maybe there are stats to back this up. I was simply saying, that in my experience I have consulted with companies using cloud computing services (ie EC2) and mysql. They are performance conscious. When they have to fire up more boxes, they pay for it immediately. When they ran into problems getting good performance out of mysql it was very easy to show them how to get better performance using postgres. (Often this was just: do the same thing in postgres, and look, it's faster!). But they also rely on replication to be able to scale. And without it they just weren't interested. Porting any project has it's own set of issues, I was speaking to the case where people are evaluating databases for a new project. I was not however trying to make any kind of statement as too how important it is as compared to any other specific feature. I was just trying to say that in my experience current trends indicate that having easy to set up replication is getting more important over time, not less, and not the same. Other features may be more important. Getting it right is certainly more important that getting it soon (for reasonable values of "soon" and "right" of course IMHO). The experience of others my be totally different, but that is mine. Just wanted to clarify what I was actually trying to say because this response seems to indicate that I didn't make certain things clear. - Rick -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Peter Eisentraut wrote: > Much of the delay and uncertainty during beta in my mind comes from the > situation that we wait for negative results and don't trust the release > until we have seen and fixed enough of them. Instead of waiting for > concrete, positive results and producing the release with confidence. Yep, that is our dilemma. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On ons, 2009-08-26 at 18:15 -0400, Tom Lane wrote: > > I think there is a lot of merit (as Andrew suggests) in running a > > production application on a beta version of the database just to see > > if anything funny happens. > > ... but here we seem to be coming out at the same place anyway. Getting > people to put their existing apps onto a beta is very productive. > We have to encourage people to do more of that while it's still beta, > instead of waiting till .0 or .1 or later. I think people should be running their applications' system tests on top of the new PostgreSQL. Just installing the application, clicking three buttons, and I-don't-have-more-time-than-this helps a little but not much. Of course many people won't have system tests, which is why this process is a problem. To pick up a current example: "Drupal system tests pass with 8.5betaN" is nice and useful. "I ran our app on 8.5betaN and didn't see any issues" is interesting, but ultimately doesn't help much. Much of the delay and uncertainty during beta in my mind comes from the situation that we wait for negative results and don't trust the release until we have seen and fixed enough of them. Instead of waiting for concrete, positive results and producing the release with confidence. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 12:03 AM, Dimitri Fontaine wrote: > Is the offering good enough? We might need to run some kind of tutorials > for users to be able to run large tests easily, and maybe think about > some newer tools allowing to compare logs of two application runs in two > database versions (capture all queries and result in a database, then > have a way to diff). Then beta testing would mean having a spare machine > where to run the magic regression test suite against some internal > application. Well, people may recall I spent a lot of time testing 8.3 before and during beta. The important words are "a lot of time", probably one month full time spread on 3 months to find *only* 3 problems Tom, Alvarro and Andrew fixed: yes one month for only 3 problems identified, reported, discussed and fixed. The problem isn't to connect your application to the database - it's the easy part: if you have a large one, you probably won't see the anomalies. The application I used for my tests is displaying every SQL query at the bottom of the page with the time spent executing the query, I was switching from the 8.1 site to the 8.3 site to check everything manually. I also got all the urls of this application (more than one million), and use a load test tool to load each page and pgFouine to grab any error from the PostgreSQL logs. Even with these information and this work, I'm pretty sure I would have missed a join problem which would have returned 2 or 3 more rows even with the time I spent working on it. My plan at the time was to develop an application which would parse the query logs from the server, replay the queries on 2 PG servers with both versions and report me the anomalies (difference in the number of rows, difference in the content, queries slower than with the old version). I haven't had the opportunity to work on the 8.4 beta test due to my daily (and often nightly) job work load but the idea is still there and IMHO it's really necessary if we want to be able to detect the problems we only discovered after the release. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Wed, Aug 26, 2009 at 11:15 PM, Tom Lane wrote: > ... but here we seem to be coming out at the same place anyway. Getting > people to put their existing apps onto a beta is very productive. > We have to encourage people to do more of that while it's still beta, > instead of waiting till .0 or .1 or later. +1 for the goal being to get users to test their applications on beta. I also wonder if we've adopted the wrong strategy with betas by stopping development during them. It seems to be the worst of both worlds in that both developers and users are unhappy. Perhaps we should fork 8.6 right away after 8.5.beta is released (I'm assuming *after* any open issues are closed) and start a commitfest with any pending patches. While we do that call for users to test 8.5.beta with their applications and wait a fixed amount of time for any bugs to turn up. That lets us have a long beta period for users to test on without stopping development. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas writes: > On Wed, Aug 26, 2009 at 5:12 PM, Peter Eisentraut wrote: >> ... Surely it's been tested before, else it would not be in >> the release, right? > I would sure hope so. Testing features individually makes a whole lot > more sense to me than testing the release as a whole. Just trying a > bunch of random stuff and seeing if anything breaks is not a very > productive activity. I beg to disagree. New features have presumably been tested, in isolation, by the original developer and reviewers. The things we tend to miss are bad side-effects on corner cases and seemingly-unrelated features. So "testing as a whole" is exactly what beta is for, to my mind. > I think there is a lot of merit (as Andrew suggests) in running a > production application on a beta version of the database just to see > if anything funny happens. ... but here we seem to be coming out at the same place anyway. Getting people to put their existing apps onto a beta is very productive. We have to encourage people to do more of that while it's still beta, instead of waiting till .0 or .1 or later. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Hi, Peter Eisentraut writes: > On ons, 2009-08-26 at 14:26 -0400, Robert Haas wrote: >> Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months >> doesn't necessarily buy you much, either. I'm good at focused >> activity - but there was nothing focused about 8.4 beta that I could >> see. Maybe we need some kind of TestFest process. > > Yeah, exactly. I can't imagine end users would know what to do during > beta. Even assuming that you have release notes at the beginning of > beta, you can't expect people to go through every item and do a formal > test for it. Surely it's been tested before, else it would not be in > the release, right? Well we all know that developpers are really bad at testing, because they tend to test for cases they though about while developping the code rather than being creative and running a full application against the overall new product. Now, it could be that what we miss is some tool suite to enable application people to test their full code against our new releases and check results, performance, plans, etc. I know about a couple of tools to get there, Tsung and Playr. And the focus is performance testing and scalability, not that it still works. Is the offering good enough? We might need to run some kind of tutorials for users to be able to run large tests easily, and maybe think about some newer tools allowing to compare logs of two application runs in two database versions (capture all queries and result in a database, then have a way to diff). Then beta testing would mean having a spare machine where to run the magic regression test suite against some internal application. Regards, -- dim Tsung: http://tsung.erlang-projects.org/ Playr: https://area51.myyearbook.com/trac.cgi/wiki/Playr PS: sql level unit testing isn't an answer here as it means the application already have the tests when entering beta. Hard sell. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Wed, Aug 26, 2009 at 5:12 PM, Peter Eisentraut wrote: > On ons, 2009-08-26 at 14:26 -0400, Robert Haas wrote: >> Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months >> doesn't necessarily buy you much, either. I'm good at focused >> activity - but there was nothing focused about 8.4 beta that I could >> see. Maybe we need some kind of TestFest process. > > Yeah, exactly. I can't imagine end users would know what to do during > beta. Even assuming that you have release notes at the beginning of > beta, you can't expect people to go through every item and do a formal > test for it. Surely it's been tested before, else it would not be in > the release, right? I would sure hope so. Testing features individually makes a whole lot more sense to me than testing the release as a whole. Just trying a bunch of random stuff and seeing if anything breaks is not a very productive activity. On the other hand, testing individual features is frequently very productive, but it's my understanding of the way PG does things that that is supposed to happen before the patch is committed. It appears to me that most of the really nasty bugs that have been found in 8.4.0 relate to one of the following three things, each of which seems to be related to multiple back-branch commits. 1. SEMI/ANTI join support. 2. running the bgwriter during recovery (infrastructure changes for recovery) 3. deadman switch Maybe some of these weren't tested well enough prior to commit? Or perhaps they're just more significant changes and therefore likely spots for rough edges. I think there is a lot of merit (as Andrew suggests) in running a production application on a beta version of the database just to see if anything funny happens. But insisting that all PG developers stop doing development to focus ONLY on that activity doesn't seem very reasonable: not everyone is well-placed to do that kind of experiment, or cares to do so. Conversely, there are many people who are NOT developers who ARE well-placed to beta test (for example, Kevin Grittner does a lot more testing than he does development, and I think there are few people on this mailing list who would argue that the quality of that testing is any less than kickass). ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Josh Berkus wrote: Tom, all, As far as the alpha releases go, I wouldn't --- I see no evidence that we have the manpower to formalize them any more than they are now. I do like the idea of trying to reach out to more beta testers and manage that phase more aggressively. Maybe if we can make something happen there, it would be possible to do the same for alphas later. Well, we need a concrete list of things to beta test if we want people to do this in any organized fashion. It's easy enough for us to say "we need beta testers"; but without telling our community *what* to test, people just download, compile, do a select * from table, and say "it works". Once we have a list, we can launch a simple app/wiki page where people can report results of their tests. Without all that, the task is too amorphous for us to expect much from the community-at-large. Actually, what I had in mind was getting people to run their applications etc. in some non-production environment on the beta. I talked to a client today and he said "sure, we have several development environments and we can put one or two on the beta and then let the developers just do their thing on it." Testing the things we know about is in a way less important than making sure nothing else got broken. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Tom, all, > As far as the alpha releases go, I wouldn't --- I see no evidence that > we have the manpower to formalize them any more than they are now. > I do like the idea of trying to reach out to more beta testers and > manage that phase more aggressively. Maybe if we can make something > happen there, it would be possible to do the same for alphas later. Well, we need a concrete list of things to beta test if we want people to do this in any organized fashion. It's easy enough for us to say "we need beta testers"; but without telling our community *what* to test, people just download, compile, do a select * from table, and say "it works". Once we have a list, we can launch a simple app/wiki page where people can report results of their tests. Without all that, the task is too amorphous for us to expect much from the community-at-large. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On ons, 2009-08-26 at 14:26 -0400, Robert Haas wrote: > Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months > doesn't necessarily buy you much, either. I'm good at focused > activity - but there was nothing focused about 8.4 beta that I could > see. Maybe we need some kind of TestFest process. Yeah, exactly. I can't imagine end users would know what to do during beta. Even assuming that you have release notes at the beginning of beta, you can't expect people to go through every item and do a formal test for it. Surely it's been tested before, else it would not be in the release, right? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
David Fetter writes: > On Wed, Aug 26, 2009 at 02:46:43PM -0400, Tom Lane wrote: >> The alpha releases as currently constituted are practically the >> exact opposite of what's being suggested here :-(. We are pushing >> them out without very much advertisement, and certainly without any >> formal program to encourage significant testing. That's more or >> less forced by the decision that alpha releases should be >> low-overhead, but I think we're unlikely to get wide-ranging test >> coverage that way. > How would you recommend that this change? As far as the alpha releases go, I wouldn't --- I see no evidence that we have the manpower to formalize them any more than they are now. I do like the idea of trying to reach out to more beta testers and manage that phase more aggressively. Maybe if we can make something happen there, it would be possible to do the same for alphas later. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Wed, Aug 26, 2009 at 02:46:43PM -0400, Tom Lane wrote: > "Matthew T. O'Connor" writes: > > Alvaro Herrera wrote: > >> This seems a good idea. Possibly pushing the betas more > >> aggresively to current users would make them tested not only by > >> PG hackers ... > > > Isn't this the purpose of the new alpha releases, at lease to some > > extent. > > The alpha releases as currently constituted are practically the > exact opposite of what's being suggested here :-(. We are pushing > them out without very much advertisement, and certainly without any > formal program to encourage significant testing. That's more or > less forced by the decision that alpha releases should be > low-overhead, but I think we're unlikely to get wide-ranging test > coverage that way. How would you recommend that this change? Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
2009/8/26 Jean-Michel Pouré : > Dear Kevin > >> So when you talk about focusing on usablility improvements you mean >> that priority should be given to supporting MySQL-specific syntax >> extensions and ensuring that there are no queries where the MySQL >> optimizer comes up with a more efficient plan than PostgreSQL? > > Yes. PostgreSQL should be able to run MySQL code quoted here: > > This is a prerequisite for people to be willing to test and adopt > PostgreSQL. People are not willing to debug frameworks like Drupal and > port them to PostgreSQL. We are quite alone and lost. Er, so we should debug Drupal for them? I find it difficult to believe that's the best use of our time. >> One concern I have is that you don't mention PostgreSQL configuration >> in your performance advice, and I seem to remember you said that you >> didn't tune your postgresql.conf file beyond boosting the >> shared_buffers setting. If that's true, you might be somewhat >> surprised with the performance improvements if you tweak just a few >> other settings. > > shared_buffer 24M. That doesn't actually respond to anything he said in that paragraph. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
"Matthew T. O'Connor" writes: > Alvaro Herrera wrote: >> This seems a good idea. Possibly pushing the betas more aggresively to >> current users would make them tested not only by PG hackers ... > Isn't this the purpose of the new alpha releases, at lease to some extent. The alpha releases as currently constituted are practically the exact opposite of what's being suggested here :-(. We are pushing them out without very much advertisement, and certainly without any formal program to encourage significant testing. That's more or less forced by the decision that alpha releases should be low-overhead, but I think we're unlikely to get wide-ranging test coverage that way. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Alvaro Herrera wrote: This seems a good idea. Possibly pushing the betas more aggresively to current users would make them tested not only by PG hackers ... Isn't this the purpose of the new alpha releases, at lease to some extent. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Dear Kevin > So when you talk about focusing on usablility improvements you mean > that priority should be given to supporting MySQL-specific syntax > extensions and ensuring that there are no queries where the MySQL > optimizer comes up with a more efficient plan than PostgreSQL? Yes. PostgreSQL should be able to run MySQL code quoted here: This is a prerequisite for people to be willing to test and adopt PostgreSQL. People are not willing to debug frameworks like Drupal and port them to PostgreSQL. We are quite alone and lost. > One concern I have is that you don't mention PostgreSQL configuration > in your performance advice, and I seem to remember you said that you > didn't tune your postgresql.conf file beyond boosting the > shared_buffers setting. If that's true, you might be somewhat > surprised with the performance improvements if you tweak just a few > other settings. shared_buffer 24M. Kind regards, Jean-Michel signature.asc Description: Ceci est une partie de message numériquement signée
Re: [HACKERS] 8.5 release timetable, again
On Wed, Aug 26, 2009 at 1:01 PM, Tom Lane wrote: > Yup. This is a huge problem and we need to deal with it somehow. At > the same time, I'm worried that our beta testing process is already > inadequate. We've found several rather embarrassing bugs in 8.4, for > instance, things that should have been found in beta IMO. Shortening > beta or encouraging people to start next-cycle development instead of > testing doesn't seem like a wise move. You can't just develop all > the time, you have to test & debug too ... Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months doesn't necessarily buy you much, either. I'm good at focused activity - but there was nothing focused about 8.4 beta that I could see. Maybe we need some kind of TestFest process. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Andrew Dunstan wrote: > Perhaps some more formalised beta program would be useful. I have > at least one client who could probably be persuaded to devote some > resources to Beta testing. Maybe we need a Beta Program co-ordinator > or some such animal. I suspect that plenty of possible beta testers > don't even know when we go beta. In some shops it would be necessary to have a date set months in advance of the start of the beta to have any chance of getting managers to assign hardware and staff for a really good test. I usually have to work in tests on my own time on whatever hardware happens to be in standby status, when I can get it. Hitting such a date would seem to require a significant change from prior releases; although the just-completed commit-fest could be taken as evidence that such a thing is possible. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Andrew Dunstan escribió: > Perhaps some more formalised beta program would be useful. I have at > least one client who could probably be persuaded to devote some > resources to Beta testing. Maybe we need a Beta Program co-ordinator > or some such animal. I suspect that plenty of possible beta testers > don't even know when we go beta. This seems a good idea. Possibly pushing the betas more aggresively to current users would make them tested not only by PG hackers ... -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Tom Lane wrote: Andrew Dunstan writes: My concern is not just with those features, but with the whole ratio of the window for new work to the total development cycle. That ratio keeps going down and the time the tree is effectively frozen to new features keeps going up. Yup. This is a huge problem and we need to deal with it somehow. At the same time, I'm worried that our beta testing process is already inadequate. We've found several rather embarrassing bugs in 8.4, for instance, things that should have been found in beta IMO. Shortening beta or encouraging people to start next-cycle development instead of testing doesn't seem like a wise move. You can't just develop all the time, you have to test & debug too ... Perhaps some more formalised beta program would be useful. I have at least one client who could probably be persuaded to devote some resources to Beta testing. Maybe we need a Beta Program co-ordinator or some such animal. I suspect that plenty of possible beta testers don't even know when we go beta. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
> Robert Haas wrote: >> I am assuming that at least Hot Standby and Streaming Replication will >> likely require two CommitFests to go from the point where they are >> seriously reviewable to actual commit. FWIW, I think that both HS and SR are special cases: if we ever see reviewable patches for them, people will probably be willing to work on them outside the CommitFest framework. We shouldn't be setting the schedule with the idea that those will only be dealt with in CFs. To my mind the CF process is for dealing with "run of the mill" patches. Andrew Dunstan writes: > My concern is not just with those features, but with the whole ratio of > the window for new work to the total development cycle. That ratio keeps > going down and the time the tree is effectively frozen to new features > keeps going up. Yup. This is a huge problem and we need to deal with it somehow. At the same time, I'm worried that our beta testing process is already inadequate. We've found several rather embarrassing bugs in 8.4, for instance, things that should have been found in beta IMO. Shortening beta or encouraging people to start next-cycle development instead of testing doesn't seem like a wise move. You can't just develop all the time, you have to test & debug too ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Andrew Dunstan wrote: > the window for new work to the total development cycle. That ratio > keeps going down and the time the tree is effectively frozen to new > features keeps going up. I'd like to see us keep the tree open as > long as possible but be much more ruthless about chopping off things > that aren't ready at the end. That way we can quickly get to a beta > and get on with the next cycle. I realise the idea is that > significant features must be submitted by the penultimate CF, but > I'm not too sure how well that's going to work in practice. That > just seems like we're relabelling things rather than a fundamental > change. At the very least my vote goes for four CFs rather than > three. Unless the community can reduce the time between the start of the last commit-fest and the release, you're limited to an average of four months of programming time per year for new features (assuming that people are observing the rules about what they should be doing during commit-fests and beta testing). If you want to move the next release back into Spring rather than Summer (which is the season in which 8.4 was released -- at least of those of us in the Northern Hemisphere), you would need to shorten that to three months for this release. Unless... Both the ruthless cutting of anything not totally ready at the end of a commit-fest, *and* reducing the time from the end of the last commit-fest to release would be needed to get that up to five months per year. We obviously don't want less testing during the beta cycle, but delaying the release while the release notes are developed at the end of the cycle seems like an obvious target for improvement. I'd bet there are others, though I don't know what they are -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Wed, Aug 26, 2009 at 12:07 PM, Andrew Dunstan wrote: > Robert Haas wrote: >> >> I am assuming that at least Hot Standby and Streaming Replication will >> likely require two CommitFests to go from the point where they are >> seriously reviewable to actual commit. So if they hit the 9/15 date, >> they should make 8.5 even with just three CommitFests. If they don't >> hit the 9/15 date, then a 3-CommitFest cycle will probably be too >> short for them to make it in. But if we schedule a fourth CommitFest >> in January in the hopes of seeing one of those patches committed, then >> ISTM we're basically speculating that the patch authors will not hit >> the 9/15 date but that they will hit an 11/15 date. > > My concern is not just with those features, but with the whole ratio of the > window for new work to the total development cycle. That ratio keeps going > down and the time the tree is effectively frozen to new features keeps going > up. I think that's a very valid concern. Against that, if release cycles become very long, then features can hit the tree more of the time, but they don't get into a released version for ages. > I'd like to see us keep the tree open as long as possible but be much > more ruthless about chopping off things that aren't ready at the end. That > way we can quickly get to a beta and get on with the next cycle I'm happy to assist with that, but recall that even after we ended CF 2008-11 another four months went by before release. That's a whole lotta time for the tree to be closed right there. > I realise > the idea is that significant features must be submitted by the penultimate > CF, but I'm not too sure how well that's going to work in practice. That > just seems like we're relabelling things rather than a fundamental change. > At the very least my vote goes for four CFs rather than three. Fair enough, more votes are good. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Robert Haas wrote: I am assuming that at least Hot Standby and Streaming Replication will likely require two CommitFests to go from the point where they are seriously reviewable to actual commit. So if they hit the 9/15 date, they should make 8.5 even with just three CommitFests. If they don't hit the 9/15 date, then a 3-CommitFest cycle will probably be too short for them to make it in. But if we schedule a fourth CommitFest in January in the hopes of seeing one of those patches committed, then ISTM we're basically speculating that the patch authors will not hit the 9/15 date but that they will hit an 11/15 date. My concern is not just with those features, but with the whole ratio of the window for new work to the total development cycle. That ratio keeps going down and the time the tree is effectively frozen to new features keeps going up. I'd like to see us keep the tree open as long as possible but be much more ruthless about chopping off things that aren't ready at the end. That way we can quickly get to a beta and get on with the next cycle. I realise the idea is that significant features must be submitted by the penultimate CF, but I'm not too sure how well that's going to work in practice. That just seems like we're relabelling things rather than a fundamental change. At the very least my vote goes for four CFs rather than three. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Jean-Michel Pouré wrote: > Kevin Grittner a écrit : >> It's not clear to me what you feel is needed. > http://drupal.org/node/559302 These look like performance issues. > http://drupal.org/node/14 These are MySQL compatibility issues. So when you talk about focusing on usablility improvements you mean that priority should be given to supporting MySQL-specific syntax extensions and ensuring that there are no queries where the MySQL optimizer comes up with a more efficient plan than PostgreSQL? One concern I have is that you don't mention PostgreSQL configuration in your performance advice, and I seem to remember you said that you didn't tune your postgresql.conf file beyond boosting the shared_buffers setting. If that's true, you might be somewhat surprised with the performance improvements if you tweak just a few other settings. You might want to see what suggestions you get from: http://pgfoundry.org/projects/configurator/ -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Aug 26, 2009, at 11:18 , Jean-Michel Pouré wrote: Web apps are 95% of PostgreSQL possible users. Where does this figure come from? Michael Glaesemann grzm seespotcode net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Jean-Michel Pouré wrote: Everytime I try a new Drupal module under PostgreSQL, I run into tiny SQL problems ranging from error to performance drop. The most problematic problem is http://drupal.org/node/559986 I strongly suspect this post badly mis-diagnoses the problem. IMHO for what I know from the porting work (I worked heavily on PHPBB3 and now Drupal), the first goal is to achieve compatibility with issues mentioned there: http://drupal.org/node/14 and add mysql compat functions in PostgreSQL core without breaking existing code. That might be your goal, but it's not the community's goal, I believe. There are already external projects for compatibility libraries. You are never going to achieve 100% compatibility. To win over MySQL, the best is to work on materialized views. There are very good articles available from hackers. Why not port to C. Materialized which which update when the data is needed would be perfect. IIRC some work was being done on materialised views. Web apps are 95% of PostgreSQL possible users. Most applications these days have a web front end. But that doesn't mean the database needs to be terribly aware of them. To the database, a web server is just another client. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
Le mercredi 26 août 2009 à 09:30 -0500, Kevin Grittner a écrit : > It's not clear to me what you feel is needed. That could mean many > things Dear Kevin, I rarely post on Hackers, so I will try to explain: * I use PostgreSQL since 1998. * I took part in the development of pgAdmin 1&2. * I love PostgreSQL and I believe MySQL sucks. Recently I was forced to use MySQL for Kdenlive.org project and the database sometimes stops responding sending nothing or crap. I believe that if you use MySQL in your company for a paid work, you can die of a heart attack. But at the same time: * I rewrote very long and tedious queries from PHPBB. Maybe 100 of them. They are now part of PhpBB3. I drove all queries below 30 ms and this enables PhpBB to scale easily. I consider this is my work. * I think Drupal queries presently have performance problems. If I wanted, I would be able to drive down Drupal web site, using a collection of simple queries on projects, forum and comments. But I don't of course. This is why I wrote http://drupal.org/node/559302 and http://drupal.org/node/14 Everytime I try a new Drupal module under PostgreSQL, I run into tiny SQL problems ranging from error to performance drop. The most problematic problem is http://drupal.org/node/559986 To fix a problem, I need to open a thread on Drupal web site and wait for the maintainer to discuss and commit. To give an example, Drupal includes a query optimizer written in PHP, which sometimes adds "DISTINCT" to queries. In the forum, some incredible query fetches 22000 rows, copies the rows to an arrays and then computes the array. This allows to display previous and next message. But we are not going to change the world of MySQL users, which believe they know SQL programming and in reality are complete beginners, who like to boast about farms and replication, just like Windows users like to collect Adobe products on DVDs and discuss with friends about them. IMHO for what I know from the porting work (I worked heavily on PHPBB3 and now Drupal), the first goal is to achieve compatibility with issues mentioned there: http://drupal.org/node/14 and add mysql compat functions in PostgreSQL core without breaking existing code. Then I can insure that 99% of MySQL compatibility problems will be behind. When this is achieve, we will be able to start education of developers. And this will take another decade. To win over MySQL, the best is to work on materialized views. There are very good articles available from hackers. Why not port to C. Materialized which which update when the data is needed would be perfect. Then we can convince Drupal hackers to add views in the schema. The trick would be that MySQL would support normal views, whereas we would also support materialized. We can do the same with nearly all available frameworks: PhpBB, etc ... Web apps are 95% of PostgreSQL possible users. Kind regards, Jean-Michel signature.asc Description: Ceci est une partie de message numériquement signée
Re: [HACKERS] 8.5 release timetable, again
Jean-Michel Pouré wrote: > focus on usability. It's not clear to me what you feel is needed. That could mean many things -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
