Re: Testing nonsense
On Tue, Nov 3, 2009 at 6:47 PM, Murray Cumming wrote: > in terms of features, usability, reliability and fun. But we also face > the risk of getting maemo.org Extras associated with beta quality > software made by geeks for geeks only, without the last mile of > polishing. Possibly off-topic, but what's the current recommended place for software without that last mile of polishing (but with prospect of getting moved to extras after widespread field use)? Extras-devel isn't it, as it brings in library updates etc (subsequently bricking your device). I'm thinking of something like Ubuntu PPA's, that build the applications against the current stable libraries. -- Ville M. Vainio http://tinyurl.com/vainio ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
I think having a bugzilla where thumbs-down == unresolved bug might help. If the UI has an actual problem it should be reported, but "invalid" or "wontfix" (yes, I know it could be abused) or any other "resolved should negate any negative karma. Also it should start at a -3 or something reasonable for applications which were approved under Diablo as they aren't new. Or maybe -5 or something for the first, but if it is an update it should not require 10 thumbs up. (I also noted broad appeal applications can easily get 10, but narrower ones might have trouble - it might be easier to require 75% or 80% positive with a small quorum instead - geometric instead of arithmetic). ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
>"Oh well.. at least it'll be as much bug free as possible when it'll >finally land Extras :)" If it could land it one day ... -- Benoît HERVIER - http://khertan.net/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
On Nov 3, 2009, at 9:59 AM, Till Harbaum wrote: > I'd just like to interject that any new process like this is going to have growing pains. You have two options to deal with the kinks that inevitably appear in an untried process, help to smooth them out (See the "QA process = bug fixing disincentive?" thread) or drop into rant- mode and succeed mostly in irritating and polarizing people while doing little to assist with improving the process. I see a lot of people picking the later method right now and find it a bit disheartening. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
On Tue, Nov 3, 2009 at 16:56, Aniello Del Sorbo wrote: >> > > I missed this point while reading it. > And it convinced me to push to Extras Testing a new release of Xournal > no matter if it loses the 7 thumbs up it already got (plus the 3 it > got for a previous version) I also think that the delay for new features is a good thing now, it gives me at least 10 days whilst Hermes goes on its way to Extras of waiting and not developing until 1am in the morning :-) 10 days off: woohoo! Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
2009/11/3 Murray Cumming : > > I don't claim to know what the aims of the testing are. But I did see > this on the main testing page: > http://wiki.maemo.org/Extras-testing#The_extras-testing_QA_queue_.26_you > " > Offering good quality community software to owners of Maemo devices is a > top priority. We have a chance to show the world that open source > software developed by community projects can match commercial software > in terms of features, usability, reliability and fun. But we also face > the risk of getting maemo.org Extras associated with beta quality > software made by geeks for geeks only, without the last mile of > polishing. > " > I missed this point while reading it. And it convinced me to push to Extras Testing a new release of Xournal no matter if it loses the 7 thumbs up it already got (plus the 3 it got for a previous version) :( Oh well.. at least it'll be as much bug free as possible when it'll finally land Extras :) Thanks. -- anidel Sent from London, Eng, United Kingdom ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
On Tue, 2009-11-03 at 15:21 +, Gary Birkett wrote: > > I totally agree, it is not part of the testing regime itself and as > long as > an app is technically capable it passes the test. > the checklist has been defined here: > > http://wiki.maemo.org/Extras-testing/QA_Checklist Assuming that you are talking about ignoring UI issues, rather than just being outraged (outraged!!) that I apparently wanted the app to be ready for Maemo _6_: I don't claim to know what the aims of the testing are. But I did see this on the main testing page: http://wiki.maemo.org/Extras-testing#The_extras-testing_QA_queue_.26_you " Offering good quality community software to owners of Maemo devices is a top priority. We have a chance to show the world that open source software developed by community projects can match commercial software in terms of features, usability, reliability and fun. But we also face the risk of getting maemo.org Extras associated with beta quality software made by geeks for geeks only, without the last mile of polishing. " So I've been a) assuming that an application can't be impressive if if doesn't use Maemo 5 widgets and generally follow the Maemo 5 UI layout. b) assuming that the application maintainer would like some feedback. (Note that the wiki is generally verbose, fragmented, repetitive and hard to navigate around. That's not because it's a wiki.) -- murr...@murrayc.com www.murrayc.com www.openismus.com ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
On Tue, 2009-11-03 at 15:59 +0100, Till Harbaum wrote: > there's another problem with the testing i am facing with gpxview: > Nonsense ratings. > GPXView got a "thumbs down" for needing lots of porting to match the > maemo6 gui. > Yes, harmattan! Why the heck should a fremantle program not be > forwarded to > extras due to the fact that it will be hard to port it to qt (which is > what that guy > is imho trying to say)? It was a typo. I meant maemo 5. -- murr...@murrayc.com www.murrayc.com www.openismus.com ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
Am Dienstag, den 03.11.2009, 17:05 +0100 schrieb ds: > An other problem are security issues: How do you think a tester could > find a security issue in such an application? It is totaly impossible, > if you do not have access to a prepared vnc server. Should we assume the > vnc server to be prepared? Should we warn a user that prepared vnc > servers are not tested, so only use thrusted ones, or ... This has been discussed here already a few days ago. See for example http://lists.maemo.org/pipermail/maemo-developers/2009-October/021861.html andre -- Andre Klapper (maemo.org bugmaster) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
I am quite unhappy with the testing, too. My package vncviewer has a blocking issue (Bugtracker field), which should be marked on the package page! http://maemo.org/packages/package_instance/view/fremantle_extras-testing_free_armel/vncviewer/0.6.3-fremantle2 Not every developer is following the dev-list all the time. To me it is not transparent, how to get into extras now. If I get ten thumb ups, because no tester checked bugtracker it gets in? Now I prepared a changed debian/control file, but if I upload to testing I loose my 9 thumb ups. It is quite complicated for a developer to test this application, as he needs a vnc server to access. Probably this is the reason, why it takes a lot of time. On the other hand it is totally to transparent, what is tested. An other problem are security issues: How do you think a tester could find a security issue in such an application? It is totaly impossible, if you do not have access to a prepared vnc server. Should we assume the vnc server to be prepared? Should we warn a user that prepared vnc servers are not tested, so only use thrusted ones, or ... I have the feeling, that the process is quite slow, without being much better than having less testers. OK, enough for now:-) Am Dienstag, den 03.11.2009, 17:32 +0200 schrieb Henrik Hedberg: > Till Harbaum wrote: > > > there's another problem with the testing i am facing with gpxview: Nonsense > > ratings. > > GPXView got a "thumbs down" for needing lots of porting to match the maemo6 > > gui. > > Yes, harmattan! Why the heck should a fremantle program not be forwarded to > > extras due to the fact that it will be hard to port it to qt (which is what > > that guy > > is imho trying to say)? > > > > I am considering to entirely ignore the test process until this > > testing/promotion thing > > has actually proven to be useful. Dealing with people that just rate > > nonsense issues is > > a) a waste of time and b) frustrating. > > In addition, testers - whether they rate nonsense issues or not - > even get positive karma! It feels little unfair. I really would like to > see a discussion about the responsibilities and ethics of a tester, and > possible procedures to make sure that a tester is behaving as expected. > > BR, > > Henrik > ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
> 2009/11/3 Till Harbaum :
>> there's another problem with the testing i am facing with gpxview:
Nonsense ratings.
>> GPXView got a "thumbs down" for needing lots of porting to match the
maemo6 gui.
>> Yes, harmattan! Why the heck should a fremantle program not be
forwarded to
>> extras due to the fact that it will be hard to port it to qt (which
is what that guy
>> is imho trying to say)?
Aniello Del Sorbo wrote:
> Well he didn't say he thunbed it down because of what he said in the
comment.
>
> Maybe he thumbed it down for a reason and ALSO commented that it'll be
> hard to adapt it
> to Maemo 6 later on.
The instructions for testers [1] says:
"Whenever you decide to vote an application down, please leave a comment
describing what issues you found. The best feedback is a bug number,
since this allow to track and discuss better the problems. Voting thumbs
down without any explanation doesn't help the developer getting better
software for you and the end users."
There is a long list of blockers ("must") for packages that
developers provide. Why are the instructions for testers just advices
("please", not even "should").
I think it should read:
"Whenever you decide to vote an application down, you MUST leave a
comment describing what issues you found. The best feedback is a bug
number, since this allow to track and discuss better the problems."
or even better (the commenting feature in packages interface is
overlapping thing with bugs.maemo.org):
"Whenever you decide to vote an application down, you MUST provide a
link to a bug report with severity major or higher."
Actually, I read some early postings about the subject (since
April). There were many good ideas (like linking into Bugzilla), but for
some reason we got this separate playground.
BR,
Henrik
[1] http://wiki.maemo.org/Extras-testing#Thumbs_Down
--
Henrik Hedberg - http://www.henrikhedberg.net/
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
Am Dienstag, den 03.11.2009, 15:59 +0100 schrieb Till Harbaum: > - Add a link to the bug tracker, so people can file appropriate bugs which > can then > be processed in a useful manner Possible already - add a Bugtracker field to the "control" file if I remember correctly. Also see http://wiki.maemo.org/Extras-testing/QA_Checklist#Lack_of_bug_reporting_database andre -- Andre Klapper (maemo.org bugmaster) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
Till Harbaum wrote: > there's another problem with the testing i am facing with gpxview: Nonsense > ratings. > GPXView got a "thumbs down" for needing lots of porting to match the maemo6 > gui. > Yes, harmattan! Why the heck should a fremantle program not be forwarded to > extras due to the fact that it will be hard to port it to qt (which is what > that guy > is imho trying to say)? > > I am considering to entirely ignore the test process until this > testing/promotion thing > has actually proven to be useful. Dealing with people that just rate nonsense > issues is > a) a waste of time and b) frustrating. In addition, testers - whether they rate nonsense issues or not - even get positive karma! It feels little unfair. I really would like to see a discussion about the responsibilities and ethics of a tester, and possible procedures to make sure that a tester is behaving as expected. BR, Henrik -- Henrik Hedberg - http://www.henrikhedberg.net/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
2009/11/3 Till Harbaum : > Hi, > > there's another problem with the testing i am facing with gpxview: Nonsense > ratings. > GPXView got a "thumbs down" for needing lots of porting to match the maemo6 > gui. > Yes, harmattan! Why the heck should a fremantle program not be forwarded to > extras due to the fact that it will be hard to port it to qt (which is what > that guy > is imho trying to say)? > > I am considering to entirely ignore the test process until this > testing/promotion thing > has actually proven to be useful. Dealing with people that just rate nonsense > issues is > a) a waste of time and b) frustrating. > > My proposals: > - Add links to the apps changelogs to the package rating page > - Add a small text telling the people what they are supposed to test (not > harmattan > gui portability!!) > - Add a link to the bug tracker, so people can file appropriate bugs which > can then > be processed in a useful manner > > Till > Well he didn't say he thunbed it down because of what he said in the comment. Maybe he thumbed it down for a reason and ALSO commented that it'll be hard to adapt it to Maemo 6 later on. -- anidel Sent from London, Eng, United Kingdom ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Testing nonsense
Till, I totally agree, it is not part of the testing regime itself and as long as an app is technically capable it passes the test. the checklist has been defined here: http://wiki.maemo.org/Extras-testing/QA_Checklist Maemo 5 offers stock icons covering most regular use cases, but developers > can use the icons they prefer as long as they respect copyrights. Broken > icons can be a major bug stopping a release to Extras *but discussions > about beauty/ugliness of a UI are out of scope in the QA process*. > I hope the testing interface can be adjusted to explain the purpose because I see a lot of ui/feature creep suggestion in the comments. Whilst the specific apps may not look/feel/perform exactly as we would like it is wrong to block access because of this. Use outside methods, discuss patches and changes with the developer for future versions but we need to work together to get as many stable apps into extras as possible. (Incidentally, in the specific example you cite, I think its a typo for the version) Gary On Tue, Nov 3, 2009 at 2:59 PM, Till Harbaum wrote: > Hi, > > there's another problem with the testing i am facing with gpxview: Nonsense > ratings. > GPXView got a "thumbs down" for needing lots of porting to match the maemo6 > gui. > Yes, harmattan! Why the heck should a fremantle program not be forwarded to > extras due to the fact that it will be hard to port it to qt (which is what > that guy > is imho trying to say)? > > I am considering to entirely ignore the test process until this > testing/promotion thing > has actually proven to be useful. Dealing with people that just rate > nonsense issues is > a) a waste of time and b) frustrating. > > My proposals: > - Add links to the apps changelogs to the package rating page > - Add a small text telling the people what they are supposed to test (not > harmattan > gui portability!!) > - Add a link to the bug tracker, so people can file appropriate bugs which > can then > be processed in a useful manner > > Till > > ___ > maemo-developers mailing list > maemo-developers@maemo.org > https://lists.maemo.org/mailman/listinfo/maemo-developers > ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Testing nonsense
Hi, there's another problem with the testing i am facing with gpxview: Nonsense ratings. GPXView got a "thumbs down" for needing lots of porting to match the maemo6 gui. Yes, harmattan! Why the heck should a fremantle program not be forwarded to extras due to the fact that it will be hard to port it to qt (which is what that guy is imho trying to say)? I am considering to entirely ignore the test process until this testing/promotion thing has actually proven to be useful. Dealing with people that just rate nonsense issues is a) a waste of time and b) frustrating. My proposals: - Add links to the apps changelogs to the package rating page - Add a small text telling the people what they are supposed to test (not harmattan gui portability!!) - Add a link to the bug tracker, so people can file appropriate bugs which can then be processed in a useful manner Till ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
