Re: [GENERAL] Postgres alpha testing docs and general test packs
2009/11/1 Peter Eisentraut : > On ons, 2009-10-28 at 14:13 +, Thom Brown wrote: >> I'm not really asking for the answer to those questions. I'm pointing >> out that it isn't clear (at least to me) how to determine what exactly >> has been fixed in order to test it. This doesn't apply to everything >> listed as some of it is quite clear, like "pg_dump/pg_restore --clean >> now drops large objects." > > You can be reasonably assured that the particular fixes have been tested > and work, unless they are explicitly documented otherwise. We don't > necessarily need more eyeballs to, say, check that the binary input > function of the xml type has *really* been fixed. Fair enough. :) It's my experience in testing that makes me want to test particular things to death. > One point of the alpha releases is to test whether nothing else has been > broken by the various fixes, new features, and refactorings. And you > can check that by running your application on top of the new database > server. It helps if you have a test suite for your application. For > example, if the fix of the binary input function of the xml type breaks > your application because it had relied on some undocumented corner case, > now would be good time to find that out. Yeah, I should realise this is just an alpha release really. Unfortunately I'd imagine that most places using PostgreSQL in production won't be participating in alpha testing at all as there's no call for it. No doubt I can find what I'm looking for, even if it's looking through the commit log. But I guess that's your point, I probably don't need to do that. At the moment Gentoo doesn't even want for format my primary partition (some DRDY error), so I still haven't had the chance to give this a go. :( Thom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
On ons, 2009-10-28 at 14:13 +, Thom Brown wrote: > > http://developer.postgresql.org/pgdocs/postgres/release-8-5.html > > Thanks Adrian. I just wasn't looking hard enough obviously :) That > list still doesn't appear to be explicit enough though as we have > "Multiple improvements in contrib/hstore, including raising limits on > keys and values". What exactly is meant by limit, what was this limit > before and what has it been raised to? > > Similarly: "Fix encoding handling in binary input function of xml > type." What was the problem before? > > And: "Allow the collection of statistics on sequences". How would > your average end-user see whether these statistics are being colelcted > on sequences? And are these statistics actually used anywhere yet? I agree some of the release note items could be written in a more useful way. But ultimately, we can't elaborate on every code fix in detail. Concentrate on the new features and try them out. They should be documented. If not, or you can't find the documentation, please report that. > I'm not really asking for the answer to those questions. I'm pointing > out that it isn't clear (at least to me) how to determine what exactly > has been fixed in order to test it. This doesn't apply to everything > listed as some of it is quite clear, like "pg_dump/pg_restore --clean > now drops large objects." You can be reasonably assured that the particular fixes have been tested and work, unless they are explicitly documented otherwise. We don't necessarily need more eyeballs to, say, check that the binary input function of the xml type has *really* been fixed. One point of the alpha releases is to test whether nothing else has been broken by the various fixes, new features, and refactorings. And you can check that by running your application on top of the new database server. It helps if you have a test suite for your application. For example, if the fix of the binary input function of the xml type breaks your application because it had relied on some undocumented corner case, now would be good time to find that out. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
Le jeudi 29 octobre 2009 à 02:13:51, Greg Smith a écrit : > On Wed, 28 Oct 2009, Thom Brown wrote: > > All we have are a summary of changes. We can find out all the > > information if we do plenty of searching of mailing lists and comparing > > old and new documentation, but obviously this can be off-putting and is > > duplicated for everyone who wants to participate in testing. > > For the last release, we had some people who updated blogs etc. with usage > examples for many of the new major features. That doesn't seem to be > happening as well for the 8.5 development. > I was thinking about this too. Lack of time, I suppose. But anyways, some blog notes are interesting for testers, but hard to find if you need to go on different blogs. All that kind of informations need to be found quite easily, and the better way to do this is to put all of them in the same place. Maybe the wiki can be such a place. > In any case, the whole process is still being worked out. I for example > and working on some instructions for doing performance regression testing > of the alpha releases. There actually is a full regression test suite > that gets runs all the time on many platforms. The point of the alphas is > actually for you to try *your* tests, not for everyone to test the same > thing. > > There is another route to get information here that might be a bit easier > than directly looking up things in the mailing lists or commit logs. > Each alpha is being generated after a CommitFest period during which > patches are commited. The web application organizing that process > provides one way to more easily find the relevant discussion leading up > that patch being applied, and many of those include better/more obvious > examples and documentation. The current alpha2 is based on the results of > https://commitfest.postgresql.org/action/commitfest_view?id=3 > I didn't think about this, but it seems to be a better way to build this kind of document than browsing the commit mailing list. I'll try to write something according to informations available in 2009-07 (https://commitfest.postgresql.org/action/commitfest_view?id=2) and 2009-09 (https://commitfest.postgresql.org/action/commitfest_view?id=3) commitfest. I'll begin the work next monday. -- Guillaume. http://www.postgresqlfr.org http://dalibo.com -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
On Wed, 28 Oct 2009, Thom Brown wrote: All we have are a summary of changes. We can find out all the information if we do plenty of searching of mailing lists and comparing old and new documentation, but obviously this can be off-putting and is duplicated for everyone who wants to participate in testing. For the last release, we had some people who updated blogs etc. with usage examples for many of the new major features. That doesn't seem to be happening as well for the 8.5 development. In any case, the whole process is still being worked out. I for example and working on some instructions for doing performance regression testing of the alpha releases. There actually is a full regression test suite that gets runs all the time on many platforms. The point of the alphas is actually for you to try *your* tests, not for everyone to test the same thing. There is another route to get information here that might be a bit easier than directly looking up things in the mailing lists or commit logs. Each alpha is being generated after a CommitFest period during which patches are commited. The web application organizing that process provides one way to more easily find the relevant discussion leading up that patch being applied, and many of those include better/more obvious examples and documentation. The current alpha2 is based on the results of https://commitfest.postgresql.org/action/commitfest_view?id=3 -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
2009/10/28 Alvaro Herrera : > > If anyone (you?) wants to step up and produce the document you request, > it'll probably be linked to. But please do not request the current > development team to work on it, because most of them are overloaded > already (or have other reasons not to). I can understand that, and I wouldn't expect our valuable developers to do all this work. I was thinking more of someone (or maybe more than 1 person) taking the role of test documenter yes, yes, I know I should get involved myself. I will look into putting something together to meet my own proposals. I'm not entirely what the result should look like for this particular project, but I'll see if I can come up with something. At least I might feel a little useful. ;) Thom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
On Wednesday 28 October 2009 3:55:02 pm Thom Brown wrote: > 2009/10/28 Adrian Klaver : > > - "Guillaume Lelarge" wrote: > >> Le mercredi 28 octobre 2009 à 15:13:06, Thom Brown a écrit : > >> > Similarly: "Fix encoding handling in binary input function of xml > >> > type." What was the problem before? > > > > See attached screen shot for one possible solution. > > In other words we need to scour the committers mailing list to hunt > for this information? This is exactly my point. Testing doesn't > appear to be well organised. In my last place of work we had a set of > requirements, technical solution design and a test guide which > instructed testers on what areas need testing. From these a test plan > was built to ensure that the requirements were met, and that the > technical solution was working as specified. In addition to this they > performed regression testing in the affected areas to ensure > everything else still worked as expected and wasn't negatively > affected by the new changes. On the database side this handled by 'make check' which runs a regression suite against the source. So this would be the first thing to do to ensure that the database is not affected by a regression when compiled in your particular environment. As to your particular needs/application things are a bit more involved as you mention below. Here to date it seems most people have scanned the change list for items that affected them and then dug deeper to get the particulars. One of the benefits/problems of Open Source ,some assembly required. > > All we have are a summary of changes. We can find out all the > information if we do plenty of searching of mailing lists and > comparing old and new documentation, but obviously this can be > off-putting and is duplicated for everyone who wants to participate in > testing. > > I'm suggesting that while this is technically sufficient, it might be > a better idea to provide a clear technical document of the changes > that have been committed. This can be seen as an opportunity to participate in the project. I am sure plenty of people would be grateful if you where to spearhead just such a document :) > > Such documentation may also potentially be reused when the final > version is released for end-users to review for any changes they might > need to make to their existing code and queries to ensure they don't > break. > > Obviously PostgreSQL has survived very well without this, but I would > expect this would help more users perform more testing. > > Thom As was mentioned in another post the whole Alpha release program is new, so it is still in the learning curve stage. My experience with the Postgres project is that most itches do get scratched. It just does not always happen as fast as everybody would like. -- Adrian Klaver akla...@comcast.net -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
Thom Brown escribió: > Obviously PostgreSQL has survived very well without this, but I would > expect this would help more users perform more testing. Keep in mind alphas are new. Last time around, we only released a test version when we were going to go to beta. And the alpha idea was accepted only because it was said that it was going to be very light on the developer team. If anyone (you?) wants to step up and produce the document you request, it'll probably be linked to. But please do not request the current development team to work on it, because most of them are overloaded already (or have other reasons not to). -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
2009/10/28 Adrian Klaver : > > > > - "Guillaume Lelarge" wrote: > >> Le mercredi 28 octobre 2009 à 15:13:06, Thom Brown a écrit : > >> > >> > Similarly: "Fix encoding handling in binary input function of xml >> > type." What was the problem before? >> > > > See attached screen shot for one possible solution. > In other words we need to scour the committers mailing list to hunt for this information? This is exactly my point. Testing doesn't appear to be well organised. In my last place of work we had a set of requirements, technical solution design and a test guide which instructed testers on what areas need testing. From these a test plan was built to ensure that the requirements were met, and that the technical solution was working as specified. In addition to this they performed regression testing in the affected areas to ensure everything else still worked as expected and wasn't negatively affected by the new changes. All we have are a summary of changes. We can find out all the information if we do plenty of searching of mailing lists and comparing old and new documentation, but obviously this can be off-putting and is duplicated for everyone who wants to participate in testing. I'm suggesting that while this is technically sufficient, it might be a better idea to provide a clear technical document of the changes that have been committed. Such documentation may also potentially be reused when the final version is released for end-users to review for any changes they might need to make to their existing code and queries to ensure they don't break. Obviously PostgreSQL has survived very well without this, but I would expect this would help more users perform more testing. Thom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
- "Guillaume Lelarge" wrote: > Le mercredi 28 octobre 2009 à 23:30:01, Adrian Klaver a écrit : > > - "Guillaume Lelarge" wrote: > > > Le mercredi 28 octobre 2009 à 15:13:06, Thom Brown a écrit : > > > > Similarly: "Fix encoding handling in binary input function of > xml > > > > type." What was the problem before? > > > > See attached screen shot for one possible solution. > > > > This solution aims at developers, not users. I mean, I can do this and > I > already do. My customers won't. > > > -- > Guillaume. > http://www.postgresqlfr.org > http://dalibo.com True, but I took it Thom was looking for a way to find answers in the meantime. This sort of comes under: 'Give a man a fish feed him for a day, teach him to fish feed him for a lifetime' :) Adrian Klaver akla...@comcast.net -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
Le mercredi 28 octobre 2009 à 23:30:01, Adrian Klaver a écrit : > - "Guillaume Lelarge" wrote: > > Le mercredi 28 octobre 2009 à 15:13:06, Thom Brown a écrit : > > > Similarly: "Fix encoding handling in binary input function of xml > > > type." What was the problem before? > > See attached screen shot for one possible solution. > This solution aims at developers, not users. I mean, I can do this and I already do. My customers won't. -- Guillaume. http://www.postgresqlfr.org http://dalibo.com -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
Le mercredi 28 octobre 2009 à 15:13:06, Thom Brown a écrit : > 2009/10/28 Adrian Klaver : > >> Entirely new features are easier to deal with though. I still would, > >> however, want something like a detailed version of Josh's post which > >> breaks down where the changes have occurred. It seems quite scattered > >> and unclear at the moment. > >> > >> Thom > > > > http://developer.postgresql.org/pgdocs/postgres/release-8-5.html > > Thanks Adrian. I just wasn't looking hard enough obviously :) That > list still doesn't appear to be explicit enough though as we have > "Multiple improvements in contrib/hstore, including raising limits on > keys and values". What exactly is meant by limit, what was this limit > before and what has it been raised to? > > Similarly: "Fix encoding handling in binary input function of xml > type." What was the problem before? > > And: "Allow the collection of statistics on sequences". How would > your average end-user see whether these statistics are being colelcted > on sequences? And are these statistics actually used anywhere yet? > > I'm not really asking for the answer to those questions. I'm pointing > out that it isn't clear (at least to me) how to determine what exactly > has been fixed in order to test it. This doesn't apply to everything > listed as some of it is quite clear, like "pg_dump/pg_restore --clean > now drops large objects." > You're completely right. But release notes never intended to be this. What you need is more a "visual" tour, and I don't think anyone did write such a thing for any PostgreSQL releases (but I may be proven wrong). I wrote something like this in french for 8.2, 8.3, and 8.4. The last two were even published in a french Linux magazine. I suppose other people from other countries do the same. The advocacy group would do a good thing if it starts working on this kind of document. I could probably work on this too. -- Guillaume. http://www.postgresqlfr.org http://dalibo.com -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
2009/10/28 Adrian Klaver : >> Entirely new features are easier to deal with though. I still would, >> however, want something like a detailed version of Josh's post which >> breaks down where the changes have occurred. It seems quite scattered >> and unclear at the moment. >> >> Thom > > http://developer.postgresql.org/pgdocs/postgres/release-8-5.html Thanks Adrian. I just wasn't looking hard enough obviously :) That list still doesn't appear to be explicit enough though as we have "Multiple improvements in contrib/hstore, including raising limits on keys and values". What exactly is meant by limit, what was this limit before and what has it been raised to? Similarly: "Fix encoding handling in binary input function of xml type." What was the problem before? And: "Allow the collection of statistics on sequences". How would your average end-user see whether these statistics are being colelcted on sequences? And are these statistics actually used anywhere yet? I'm not really asking for the answer to those questions. I'm pointing out that it isn't clear (at least to me) how to determine what exactly has been fixed in order to test it. This doesn't apply to everything listed as some of it is quite clear, like "pg_dump/pg_restore --clean now drops large objects." Thanks Thom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
On Wednesday 28 October 2009 6:46:13 am Thom Brown wrote: > 2009/10/28 Grzegorz Jaśkiewicz : > > have you seen that one: > > http://it.toolbox.com/blogs/database-soup/alpha2-is-out-and-we-need-you-t > >o-test-it-35032?rss=1 ? > > That's partly why I was asking. It mentions the areas where the > changes have occurred, but not necessarily the changes themselves. An > example of this is hstore. There are mentions of improvements and > issues being eliminated, but these haven't been specified, not even in > the documentation. I'm not sure how to test whatever change has gone > in. I could open 8.4 and 8.5 documentation for that same page and > flip between the two until I find a difference, but even if I do that > and find changes, I doubt that covers what the fixes are. I'd want > scenarios that were problematic in 8.4 that are not so in 8.5. > > Entirely new features are easier to deal with though. I still would, > however, want something like a detailed version of Josh's post which > breaks down where the changes have occurred. It seems quite scattered > and unclear at the moment. > > Thom http://developer.postgresql.org/pgdocs/postgres/release-8-5.html -- Adrian Klaver akla...@comcast.net -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
2009/10/28 Grzegorz Jaśkiewicz : > have you seen that one: > http://it.toolbox.com/blogs/database-soup/alpha2-is-out-and-we-need-you-to-test-it-35032?rss=1 > ? That's partly why I was asking. It mentions the areas where the changes have occurred, but not necessarily the changes themselves. An example of this is hstore. There are mentions of improvements and issues being eliminated, but these haven't been specified, not even in the documentation. I'm not sure how to test whatever change has gone in. I could open 8.4 and 8.5 documentation for that same page and flip between the two until I find a difference, but even if I do that and find changes, I doubt that covers what the fixes are. I'd want scenarios that were problematic in 8.4 that are not so in 8.5. Entirely new features are easier to deal with though. I still would, however, want something like a detailed version of Josh's post which breaks down where the changes have occurred. It seems quite scattered and unclear at the moment. Thom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Postgres alpha testing docs and general test packs
have you seen that one: http://it.toolbox.com/blogs/database-soup/alpha2-is-out-and-we-need-you-to-test-it-35032?rss=1 ?
[GENERAL] Postgres alpha testing docs and general test packs
Hi, Are there any test guides/plans generated for alpha releases, or are such things only distributed to other developers? I've seen postings which mention what the new features are, and links to documentation and other postings as to what it can do, but no single page outlining the changes together. And are there any test packs which can be run against each release to ensure everything still functions as normal? What I mean is it would run through individual tests, like performing an update, checking whether the update has applied, and returning pass if successful, and fail otherwise. Such tests should be inherently massive to match the feature set of PostgreSQL, but could be built up over time if it doesn't already exist. Would there be any value in such a thing, or is this generally not really a problem that needs solving? Obviously real-world testing is needed to see how it works in a realistic scenario, so I'm not suggesting that's any less important. Thanks Thom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
