Re: QA from extras-devel to extras-testing

2009-05-13 Thread Niels Breet
Niels, that sounds like a sage proposal to the initial uploading, what however happens when a package has gone through all this and been uploaded and active for a while with no real problems. If after a period of time reports start to come in of problems, would it be wise to put in place

Re: QA from extras-devel to extras-testing

2009-05-12 Thread Graham Cobb
On Tuesday 12 May 2009 00:06:09 Joseph Charpak wrote: And maybe if you released it, a few of those thousands would help you locate the problem. :) That brings up another good point. If I had a team of people and reasonable confidence that I could work quickly to resolve the problem then maybe

Re: QA from extras-devel to extras-testing

2009-05-12 Thread Jeremiah Foster
On May 12, 2009, at 11:39, Graham Cobb wrote: On Tuesday 12 May 2009 00:06:09 Joseph Charpak wrote: And maybe if you released it, a few of those thousands would help you locate the problem. :) That brings up another good point. If I had a team of people and reasonable confidence that I

Re: QA from extras-devel to extras-testing

2009-05-12 Thread Quim Gil
Hi! ext Graham Cobb wrote: On Monday 11 May 2009 17:58:46 Quim Gil wrote: Also, what if this is a brand new application (e.g. liqbase, last year) -- how does the developer recruit beta testers? It's extras-testing who needs to recrit betatesters, not a specific app. Imagine 200 people

Re: QA from extras-devel to extras-testing

2009-05-12 Thread Frank Banul
Just for thought since I uploaded my app to the fremantle autobuilder last night: The Tabletbridge application connects to Roku Soundbridge devices. It's pretty useless if you don't have a Soundbridge on your network. I'm sure there are similar applications that are not self contained. Of the

Re: QA from extras-devel to extras-testing

2009-05-12 Thread Niels Breet
Just for thought since I uploaded my app to the fremantle autobuilder last night: The Tabletbridge application connects to Roku Soundbridge devices. It's pretty useless if you don't have a Soundbridge on your network. I'm sure there are similar applications that are not self contained. Of

Re: QA from extras-devel to extras-testing

2009-05-12 Thread gary liquid
Niels, that sounds like a sage proposal to the initial uploading, what however happens when a package has gone through all this and been uploaded and active for a while with no real problems. If after a period of time reports start to come in of problems, would it be wise to put in place

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Jeremiah Foster
On May 10, 2009, at 2:02, Graham Cobb wrote: On Tuesday 05 May 2009 16:30:41 Quim Gil wrote: ext Jeremiah Foster wrote: On May 5, 2009, at 15:41, Quim Gil wrote: - We need to know when an application in extras-testing is ready for end users. After going through a policy check and two

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Sebastian 'CrashandDie' Lauwers
On Mon, May 11, 2009 at 10:42 AM, Jeremiah Foster jerem...@jeremiahfoster.com wrote: There is no other way. Here's why;        1. Too many packages - humans don't scale well.        2. The promotion policy needs to be tested by an automated process with proper regression and unit tests - this

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Jeremiah Foster
On May 11, 2009, at 12:43, Sebastian 'CrashandDie' Lauwers wrote: On Mon, May 11, 2009 at 10:42 AM, Jeremiah Foster jerem...@jeremiahfoster.com wrote: There is no other way. Here's why; 1. Too many packages - humans don't scale well. 2. The promotion policy needs to be tested

Re: QA from extras-devel to extras-testing

2009-05-11 Thread gary liquid
automated unit testing will not find all bugs or problems in perfectly packages software. manual human testing will not find all bugs or problems in perfectly packaged software. a combination of both involving common sense and prior history should minimize the risk of problems though. gary On

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Sebastian 'CrashandDie' Lauwers
(Clicked the wrong button and sent it to Jeremiah only first) On Mon, May 11, 2009 at 11:57 AM, Jeremiah Foster jerem...@jeremiahfoster.com wrote: Name one public company in the Fortune 500 that does not employ automated testing when writing software. It is impossible to do the whole testing

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Quim Gil
ext gary liquid wrote: automated unit testing will not find all bugs or problems in perfectly packages software. manual human testing will not find all bugs or problems in perfectly packaged software. a combination of both involving common sense and prior history should minimize the risk

Re: QA from extras-devel to extras-testing

2009-05-11 Thread gary liquid
absolutely quim, I felt nervous being able to push the first release of liqbase from -devel to extras all on my own, I initially thought back then these steps being discussed now were already in place and that I couldnt promote my own package. i do think there should be the flexibility of a

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Graham Cobb
On Monday 11 May 2009 15:13:05 Quim Gil wrote: 1. Developer pushes packages from extras-devel to extras testing. 2. Automated testing does the checks. 2a. If the test fails, the packages don't get to extras-testing. 2b. If the test is successful, the packages go to extras testing. 3. In

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Quim Gil
Hi, ext Graham Cobb wrote: Not sure about the requirement to have Nitro installed. In particular, what happens to the (potentially large number) of people using maemo-testing who do not have Nitro installed? What happens if none of the people who want to beta test this package have Nitro

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Eero Tamminen
Hi, ext Graham Cobb wrote: 3. In extras-testing the betatesters put the software into stress, equipped with Nitro (crash reporter installed in their devices) plus whatever tools they can use voluntarily. Not sure about the requirement to have Nitro installed. The name of this is Crash

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Graham Cobb
On Monday 11 May 2009 17:58:46 Quim Gil wrote: Also, what if this is a brand new application (e.g. liqbase, last year) -- how does the developer recruit beta testers? It's extras-testing who needs to recrit betatesters, not a specific app. Imagine 200 people getting their 'betatesters

Re: QA from extras-devel to extras-testing

2009-05-11 Thread Joseph Charpak
On Mon, 2009-05-11 at 19:07 +0100, Graham Cobb wrote: If this app was going to be downloaded by 50 people it probably wouldn't be worth worrying about this problem. But close to 40,000 have downloaded the current version of gpe-calendar and if I release this at least a few thousand

Re: QA from extras-devel to extras-testing

2009-05-09 Thread Graham Cobb
On Tuesday 05 May 2009 16:30:41 Quim Gil wrote: ext Jeremiah Foster wrote: On May 5, 2009, at 15:41, Quim Gil wrote: - We need to know when an application in extras-testing is ready for end users. After going through a policy check and two weeks of power users' tests? 2 weeks

Re: QA from extras-devel to extras-testing

2009-05-06 Thread Marius Vollmer
ext Murray Cumming murr...@murrayc.com writes: This separation of code and packaging (.diff.gz and .orig.tar.gz.) is IMO extremely important for Maemo and should be actively encouraged by both Nokia and the community processes. Downstream projects will thank us for it, i imagine Yes, I

Re: QA from extras-devel to extras-testing

2009-05-06 Thread Murray Cumming
On Wed, 2009-05-06 at 19:09 +0300, Marius Vollmer wrote: We have always known in the back of our minds that we should separate packaging from upstream development, but it was always too much trouble without sufficient gain if you are always making 'upstream' and package releases at the same

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Quim Gil
Hi, Let me start again because I don't think this downstream-upstream relationship is relevant to define hhis extras QA process. Proof: if an application in extras has been demoted because of a severe bug in a library it's the maintainer of that app the main responsible of finding a solution.

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Quim Gil
ext Murray Cumming wrote: On Mon, 2009-05-04 at 12:06 +0200, Jeremiah Foster wrote: [snip] And if someone does not want to create a bug tracker, for whatever reason, how can we convince them not to open their own repo if Maemo rejects their package? maemo.org extras would reject

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Jeremiah Foster
On May 5, 2009, at 15:41, Quim Gil wrote: Hi, Let me start again because I don't think this downstream-upstream relationship is relevant to define hhis extras QA process. Proof: if an application in extras has been demoted because of a severe bug in a library it's the maintainer of

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Quim Gil
ext Jeremiah Foster wrote: On May 5, 2009, at 15:41, Quim Gil wrote: The problem is that many package maintainers don't know the programming language of the software they are packaging. If you are packaging something written in erlang you will not be able to quickly fix bugs in that

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Jeremiah Foster
On May 5, 2009, at 17:30, Quim Gil wrote: ext Jeremiah Foster wrote: On May 5, 2009, at 15:41, Quim Gil wrote: The problem is that many package maintainers don't know the programming language of the software they are packaging. If you are packaging something written in erlang you will not

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Ian
Hi This is ideal, at least in theory. The problem is that many package maintainers don't know the programming language of the software they are packaging. If you are packaging something written in erlang you will not be able to quickly fix bugs in that package if you don't know erlang. This

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Quim Gil
ext Murray Cumming wrote: On Sun, 2009-05-03 at 20:15 +0200, Jeremiah Foster wrote: Who is going to build all of this infrastructure? For the voting stuff, I have no idea. Maemo/Nokia wants it so I guess they will make it happen. Hopefully the maemo.org team and the community council also

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Murray Cumming
On Tue, 2009-05-05 at 16:40 +0100, Ian wrote: Hi This is ideal, at least in theory. The problem is that many package maintainers don't know the programming language of the software they are packaging. If you are packaging something written in erlang you will not be able to quickly fix

Re: QA from extras-devel to extras-testing

2009-05-05 Thread Ian
Hi, This separation of code and packaging (.diff.gz and .orig.tar.gz.) is IMO extremely important for Maemo and should be actively encouraged by both Nokia and the community processes. Downstream projects will thank us for it, i imagine Yes, I wish that Nokia projects such as hildon stuck

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Dave Neary
Hi, Jeremiah Foster wrote: On May 2, 2009, at 16:11, Glen Ditchfield wrote: If packages don't have to have bug trackers, then QA will have to build some infrastructure to keep track of positive and negative votes, and the packager's comments on the negative votes, and declarations that the

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Jeremiah Foster
On May 4, 2009, at 11:11, Dave Neary wrote: Hi, Jeremiah Foster wrote: On May 2, 2009, at 16:11, Glen Ditchfield wrote: If packages don't have to have bug trackers, then QA will have to build some infrastructure to keep track of positive and negative votes, and the packager's comments on

Re: QA from extras-devel to extras-testing

2009-05-04 Thread David Greaves
Jeremiah Foster wrote: Good point. But haven't we avoided the issue of requiring that packages have bug trackers, and use them for QA? How is one going to enforce bug trackers on applications submitted to Maemo? And if someone does not want to create a bug tracker, for whatever reason,

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Andre Klapper
Am Montag, den 04.05.2009, 12:06 +0200 schrieb Jeremiah Foster: On May 4, 2009, at 11:11, Dave Neary wrote: Isn't this bugs.maemo.org? Is there a good reason not to use this for Maemo packaging issues? I think that is a fine idea. Do people know that it can be used for this sort of

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Dave Neary
Hi, Andre Klapper wrote: However packaging is not a product, but one aspect of a product, so I'd prefer to see bug reports filed against the specific product. In general products should (must?) have a bugtracker - otherwise I consider the developers/maintainers to not be interested in user

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Murray Cumming
On Sun, 2009-05-03 at 20:15 +0200, Jeremiah Foster wrote: Who is going to build all of this infrastructure? For the voting stuff, I have no idea. Maemo/Nokia wants it so I guess they will make it happen. Won't this type of requirement help create separate private repos? You mean the

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Murray Cumming
On Mon, 2009-05-04 at 12:06 +0200, Jeremiah Foster wrote: [snip] And if someone does not want to create a bug tracker, for whatever reason, how can we convince them not to open their own repo if Maemo rejects their package? We can't. But then it probably won't be as easy for people to

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Murray Cumming
On Mon, 2009-05-04 at 13:57 +0200, Dave Neary wrote: Hi, Andre Klapper wrote: However packaging is not a product, but one aspect of a product, so I'd prefer to see bug reports filed against the specific product. In general products should (must?) have a bugtracker - otherwise I consider

Re: QA from extras-devel to extras-testing

2009-05-04 Thread Jeremiah Foster
On May 4, 2009, at 15:04, Murray Cumming wrote: On Mon, 2009-05-04 at 13:57 +0200, Dave Neary wrote: Hi, Andre Klapper wrote: However packaging is not a product, but one aspect of a product, so I'd prefer to see bug reports filed against the specific product. In general products

Re: QA from extras-devel to extras-testing

2009-05-03 Thread Jeremiah Foster
On Apr 30, 2009, at 21:46, Andrew Zabolotny wrote: From Thu, 30 Apr 2009 12:13:33 +0300 Quim Gil quim@nokia.com wrote: The question is how to check and enforce them. What can be automated and what can be evaluated via testers feedback. Since Maemo is a community project, QA could be

Re: QA from extras-devel to extras-testing

2009-05-03 Thread Jeremiah Foster
On May 1, 2009, at 15:06, Matan Ziv-Av wrote: On Thu, 30 Apr 2009, Quim Gil wrote: What we shouldn't do is to create extras-testing or extras and let packages jump in without the QA process in place. Taking a buggy package out because of a new policy is going to be much more complicated.

Re: QA from extras-devel to extras-testing

2009-05-03 Thread Jeremiah Foster
On May 2, 2009, at 16:11, Glen Ditchfield wrote: On Thu, 2009-04-30 at 22:53 -0400, Ryan Abel wrote: Assuming the package has a bug tracker, sure. On Saturday 02 May 2009 08:14:35 Murray Cumming wrote: That seems like a good requirement in general. If packages don't have to have bug

Re: QA from extras-devel to extras-testing

2009-05-02 Thread Murray Cumming
On Thu, 2009-04-30 at 22:53 -0400, Ryan Abel wrote: Assuming the package has a bug tracker, sure. That seems like a good requirement in general. -- murr...@murrayc.com www.murrayc.com www.openismus.com ___ maemo-developers mailing list

Re: QA from extras-devel to extras-testing

2009-05-02 Thread Glen Ditchfield
On Thu, 2009-04-30 at 22:53 -0400, Ryan Abel wrote: Assuming the package has a bug tracker, sure. On Saturday 02 May 2009 08:14:35 Murray Cumming wrote: That seems like a good requirement in general. If packages don't have to have bug trackers, then QA will have to build some infrastructure

Re: QA from extras-devel to extras-testing

2009-05-01 Thread Matan Ziv-Av
On Thu, 30 Apr 2009, Quim Gil wrote: What we shouldn't do is to create extras-testing or extras and let packages jump in without the QA process in place. Taking a buggy package out because of a new policy is going to be much more complicated. Let's see: Just a few months ago there was a

Re: QA from extras-devel to extras-testing

2009-05-01 Thread Attila Csipa
On Thursday 30 April 2009 11:13:33 Quim Gil wrote: These are many items but kind of makes sense to have them, isn't it. The question is how to check and enforce them. What can be automated and what can be evaluated via testers feedback. The first question is how to jump from devel to testing,

QA from extras-devel to extras-testing

2009-04-30 Thread Quim Gil
Hi, The Fremantle beta SDK is out and now the real time starts for application developers targeting Maemo 5. The plan is to have the maemo.org extras repository active by default in the Application Manager. This means that Maemo 5 users opening the AM for the first time will see already the apps

Re: QA from extras-devel to extras-testing

2009-04-30 Thread Andrew Flegg
On Thu, Apr 30, 2009 at 20:46, Andrew Zabolotny z...@homelink.ru wrote: From Thu, 30 Apr 2009 12:13:33 +0300 Quim Gil quim@nokia.com wrote: The question is how to check and enforce them. What can be automated and what can be evaluated via testers feedback. Since Maemo is a community

Re: QA from extras-devel to extras-testing

2009-04-30 Thread Anderson Lizardo
On Thu, Apr 30, 2009 at 5:13 AM, Quim Gil quim@nokia.com wrote: - Their info in the app manager is complete (icon, summary, URL to project, updates info). What about packages that are only uploaded to extras to satisfy build/runtime dependencies and that should not list in the application

Re: QA from extras-devel to extras-testing

2009-04-30 Thread Glen Ditchfield
Andrew Zabolotny wrote: If package gets at least one negative vote, it's out of the queue for extras. I think we should require that the negative vote must be linked to a bug report with severity major or higher. QA could release the package by reducing its severity.

Re: QA from extras-devel to extras-testing

2009-04-30 Thread Ryan Abel
On Apr 30, 2009, at 10:15 PM, Glen Ditchfield wrote: Andrew Zabolotny wrote: If package gets at least one negative vote, it's out of the queue for extras. I think we should require that the negative vote must be linked to a bug report with severity major or higher. QA could release the