RE: How to use extras-testing correctly?

[tero.kojo]

> -Original Message-
> From: ext Anderson Lizardo [mailto:anderson.liza...@openbossa.org] 
> Sent: 24 September, 2009 21:58
> To: Kojo Tero (Nokia-D/Helsinki)
> Cc: ni...@maemo.org; ani...@gmail.com; maemo-developers@maemo.org
> Subject: Re: How to use extras-testing correctly?
> 
> On Thu, Sep 24, 2009 at 1:43 PM,   wrote:
> > Also I'll take the time to ask Nokia testing to look at the tooling 
> > issue. I would like to have some nice set of tools for testing the 
> > measurable aspects of applications (like battery usage as 
> Igor pointed out).
> 
> For battery life analysis, maybe Nokia Energy Profiler for 
> Maemo might help. From this thread comment, it seems it will 
> come with N900 too:
> 
> http://talk.maemo.org/showpost.php?p=324693&postcount=231

Just had a talk over here about this and the Energy Profiler could come out as 
a closed source app (sorry, opening it looks like a no go, things that close to 
the battery are problematic). Some checks needed, but I'll continue bugging 
people about it.

As for other tools, the tools repository used for the SDK contains a really 
nice set of tools that can be run on-device too. And the repository is allready 
there, so it's only a question of adding it in application manager. I'll do 
some wiki updating to the extras testing article to get that info up-to-date.

Tero

> Regards,
> --
> Anderson Lizardo
> OpenBossa Labs - INdT
> Manaus - Brazil
> 
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[Benoît HERVIER]

Yeah ... come back to the old time where every developpers create his
own repository. :)


2009/9/28 gary liquid :
>
>
> On Fri, Sep 25, 2009 at 5:29 PM,  wrote:
>>
>> - Original message -
>> > the apps in maemo extras *should* be trusted because we, the community,
>> > trust
>> > the developers who put them there.
>>
>> Gary, I trust the community, but I really want to be sure.
>>
>> It is also because I like the community so much that I want to keep extras
>> a safe place. For some new users it will be the first point of contact to
>> OSS. If that contact is good, more people will find the community and more
>> will join.
>
> nobody can anonymously upload to extras without first applying.
> from a community perspective, there is already a feeling of being vetted
> prior to getting upload rights.
>>
>> > it would take 1 bad report to have the software removed from extras.
>> >
>> > its a worrying scenario for some people,  but this isnt the wild west
>> > and like
>> > all trust based mechanisms, people in the community are given rights to
>> > upload
>> > hopefully based on their standing.
>>
>> That would be one form of security I would be ok with.
>> But screening people (karma or participation or whatever) for the right to
>> upload is even more questionable than having a team of testers go through
>> the apps. Everyone has to have the right to right to put their stuff to
>> devel and testing.
>
> as said, there is already an application stage.
> if the community mindset is there of vetting, no matter how vague, it helps.
>>
>> > There are many steps along the way to being involved in the community
>> > and i do
>> > not see why an individual would be nefarious enough to go through all
>> > those just
>> > to infect a few machines.
>> >
>> > people are given rights and responsibilities and mechanisms are in place
>> > to
>> > hopefully prevent an incident such as you are describing.
>>
>> Pretty much so. But I don't want to risk even a single case however
>> unlikely it is.
>
> *nod* this is a common goal.
>>
>> > it falls on each and every one of us to maintain that trust.
>>
>> It is about trust, but there is the question of security too.
>>
>> I hope the solution that is now implemented is one that works, but as
>> always, if practise shows that it needs to be rethought, then we will.
>
> yes, testing is the further step and should help to prevent even the most
> determined of individuals.
> it is rare to see applications coming through maemo.org where there isn't
> community participation at some level
>
> gary
>>
>> Tero
>>
>> Tero
>>
>> > gary
>> >
>> >
>> >
>> >
>> > On Fri, Sep 25, 2009 at 3:40 PM, David Greaves
>> >  wrote:
>> >
>> > tero.k...@nokia.com wrote:
>> > > - Original message -
>> > > >
>> >
>> > > > I realise this is a slightly different question (hence the new
>> > > > subject)
>> > > >
>> > > > OK, say I have an evil twin who wants to attack ('own') a lot of
>> > > > Nokia
>> > > N900
>> > > > devices. How do I do this?
>> > >
>> > > I hope that was retorical. Tell your evil twin to do something
>> > > usefull.
>> >
>> >
>> > Err, no it wasn't retorical; it was hypothetical though in case you were
>> > worried.
>> >
>> > It's more about being responsible :)
>> > Actually it is very late in the day to be asking... but hey, it sounds
>> > like a
>> > topic worth raising.
>> >
>> > > > Does extras-testing factor into this?
>> > >
>> > > At least so that I would prefer maemo.org extras to be clean from
>> > > malware. It is much easier to promote it in Nokia internally when
>> > > extras
>> > > contains good software.
>> >
>> >
>> > I agree 100% ... all it takes is one example of malware introduced into
>> > an OSS
>> > product and we (and Nokia) could lose a lot of credibility.
>> >
>> > I wonder how much that could be worth to some people? Maybe worth a
>> > deliberate
>> > attack? Maybe someone is playing a longer game?
>> >
>> > I just hope we are not planning on taking the "cross your fingers and
>> > toes
>> > *REALLY HARD* and hope everyone is nice to us" approach to security ;)
>> >
>> > Discuss...
>> >
>> > David
>> >
>> >
>> > --
>> > "Don't worry, you'll be fine; I saw it work in a cartoon once..."
>> >
>> >
>> >
>> > ___
>> > maemo-developers mailing list
>> > maemo-developers@maemo.org
>> > https://lists.maemo.org/mailman/listinfo/maemo-developers
>> >
>> >
>> >
>> >
>> >
>> >
>>
>
>
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
>



-- 
Benoît HERVIER - http://khertan.net/
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[gary liquid]

On Fri, Sep 25, 2009 at 5:29 PM,  wrote:

>  - Original message -
> > the apps in maemo extras *should* be trusted because we, the community,
> trust
> > the developers who put them there.
>
> Gary, I trust the community, but I really want to be sure.
>
> It is also because I like the community so much that I want to keep extras
> a safe place. For some new users it will be the first point of contact to
> OSS. If that contact is good, more people will find the community and more
> will join.
>

nobody can anonymously upload to extras without first applying.
from a community perspective, there is already a feeling of being vetted
prior to getting upload rights.

>
> > it would take 1 bad report to have the software removed from extras.
> >
> > its a worrying scenario for some people,  but this isnt the wild west and
> like
> > all trust based mechanisms, people in the community are given rights to
> upload
> > hopefully based on their standing.
>
> That would be one form of security I would be ok with.
> But screening people (karma or participation or whatever) for the right to
> upload is even more questionable than having a team of testers go through
> the apps. Everyone has to have the right to right to put their stuff to
> devel and testing.
>

as said, there is already an application stage.
if the community mindset is there of vetting, no matter how vague, it helps.


>
> > There are many steps along the way to being involved in the community and
> i do
> > not see why an individual would be nefarious enough to go through all
> those just
> > to infect a few machines.
> >
> > people are given rights and responsibilities and mechanisms are in place
> to
> > hopefully prevent an incident such as you are describing.
>
> Pretty much so. But I don't want to risk even a single case however
> unlikely it is.
>

*nod* this is a common goal.

>
> > it falls on each and every one of us to maintain that trust.
>
> It is about trust, but there is the question of security too.
>
> I hope the solution that is now implemented is one that works, but as
> always, if practise shows that it needs to be rethought, then we will.
>

yes, testing is the further step and should help to prevent even the most
determined of individuals.
it is rare to see applications coming through maemo.org where there isn't
community participation at some level

gary

>
> Tero
>
> Tero
>
> > gary
> >
> >
> >
> >
> > On Fri, Sep 25, 2009 at 3:40 PM, David Greaves
> >  wrote:
> >
> > tero.k...@nokia.com wrote:
> > > - Original message -
> > > >
> >
> > > > I realise this is a slightly different question (hence the new
> subject)
> > > >
> > > > OK, say I have an evil twin who wants to attack ('own') a lot of
> Nokia
> > > N900
> > > > devices. How do I do this?
> > >
> > > I hope that was retorical. Tell your evil twin to do something usefull.
>
> >
> >
> > Err, no it wasn't retorical; it was hypothetical though in case you were
> worried.
> >
> > It's more about being responsible :)
> > Actually it is very late in the day to be asking... but hey, it sounds
> like a
> > topic worth raising.
> >
> > > > Does extras-testing factor into this?
> > >
> > > At least so that I would prefer maemo.org extras to be clean from
> > > malware. It is much easier to promote it in Nokia internally when
> extras
> > > contains good software.
> >
> >
> > I agree 100% ... all it takes is one example of malware introduced into
> an OSS
> > product and we (and Nokia) could lose a lot of credibility.
> >
> > I wonder how much that could be worth to some people? Maybe worth a
> deliberate
> > attack? Maybe someone is playing a longer game?
> >
> > I just hope we are not planning on taking the "cross your fingers and
> toes
> > *REALLY HARD* and hope everyone is nice to us" approach to security ;)
> >
> > Discuss...
> >
> > David
> >
> >
> > --
> > "Don't worry, you'll be fine; I saw it work in a cartoon once..."
> >
> >
> >
> > ___
> > maemo-developers mailing list
> > maemo-developers@maemo.org
> > https://lists.maemo.org/mailman/listinfo/maemo-developers
> >
> >
> >
> >
> >
> >
>
>
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[Marius Vollmer]

"Kojo Tero (Nokia-D/Helsinki)"  writes:

> But screening people (karma or participation or whatever) for the
> right to upload is even more questionable than having a team of
> testers go through the apps. Everyone has to have the right to right
> to put their stuff to devel and testing.

Hmm, I think there is a fine line here worth emphasizing.

Everyone should have the right to _apply_ for upload rights and be
considered equal.  I.e., there should be no discrimination based on
irrelevant things.  For example, we should not exclude people just
because they are on IPv6 or only have a PowerPC Mac.

But not everyone has a natural right to upload to Maemo Extras.  It is
OK to screen people and demand certain references.  Maemo Extras does
not need to be free for all.

I think this is a ethically sound position, since Maemo Extras is not
the only channel for getting software to the devices.  Software
authors do not, technically, need the help or approval of maemo.org to
get into contact with their users.

If we get the balance wrong between being useful and protecting from
harm, other can show how to do it right.  You could say that the market
place will keep us honest by allowing healthy competition.
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[tero.kojo]

- Original message -
> the apps in maemo extras *should* be trusted because we, the community, trust
> the developers who put them there.

Gary, I trust the community, but I really want to be sure.

It is also because I like the community so much that I want to keep extras a 
safe place. For some new users it will be the first point of contact to OSS. If 
that contact is good, more people will find the community and more will join.

> it would take 1 bad report to have the software removed from extras.
>
> its a worrying scenario for some people,  but this isnt the wild west and like
> all trust based mechanisms, people in the community are given rights to upload
> hopefully based on their standing.

That would be one form of security I would be ok with.
But screening people (karma or participation or whatever) for the right to 
upload is even more questionable than having a team of testers go through the 
apps. Everyone has to have the right to right to put their stuff to devel and 
testing.

> There are many steps along the way to being involved in the community and i do
> not see why an individual would be nefarious enough to go through all those 
> just
> to infect a few machines.
>
> people are given rights and responsibilities and mechanisms are in place to
> hopefully prevent an incident such as you are describing.

Pretty much so. But I don't want to risk even a single case however unlikely it 
is.

> it falls on each and every one of us to maintain that trust.

It is about trust, but there is the question of security too.

I hope the solution that is now implemented is one that works, but as always, 
if practise shows that it needs to be rethought, then we will.

Tero

Tero

> gary
>
>
>
>
> On Fri, Sep 25, 2009 at 3:40 PM, David Greaves
> mailto:da...@dgreaves.com>> wrote:
>
> tero.k...@nokia.com wrote:
> > - Original message -
> > >
>
> > > I realise this is a slightly different question (hence the new subject)
> > >
> > > OK, say I have an evil twin who wants to attack ('own') a lot of Nokia
> > N900
> > > devices. How do I do this?
> >
> > I hope that was retorical. Tell your evil twin to do something usefull.
>
>
> Err, no it wasn't retorical; it was hypothetical though in case you were 
> worried.
>
> It's more about being responsible :)
> Actually it is very late in the day to be asking... but hey, it sounds like a
> topic worth raising.
>
> > > Does extras-testing factor into this?
> >
> > At least so that I would prefer maemo.org extras to be clean from
> > malware. It is much easier to promote it in Nokia internally when extras
> > contains good software.
>
>
> I agree 100% ... all it takes is one example of malware introduced into an OSS
> product and we (and Nokia) could lose a lot of credibility.
>
> I wonder how much that could be worth to some people? Maybe worth a deliberate
> attack? Maybe someone is playing a longer game?
>
> I just hope we are not planning on taking the "cross your fingers and toes
> *REALLY HARD* and hope everyone is nice to us" approach to security ;)
>
> Discuss...
>
> David
>
>
> --
> "Don't worry, you'll be fine; I saw it work in a cartoon once..."
>
>
>
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
>
>
>
>
>

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[gary liquid]

the apps in maemo extras *should* be trusted because we, the community,
trust the developers who put them there.

it would take 1 bad report to have the software removed from extras.

its a worrying scenario for some people,  but this isnt the wild west and
like all trust based mechanisms, people in the community are given rights to
upload hopefully based on their standing.

There are many steps along the way to being involved in the community and i
do not see why an individual would be nefarious enough to go through all
those just to infect a few machines.

people are given rights and responsibilities and mechanisms are in place to
hopefully prevent an incident such as you are describing.

it falls on each and every one of us to maintain that trust.

gary




On Fri, Sep 25, 2009 at 3:40 PM, David Greaves  wrote:

> tero.k...@nokia.com wrote:
> > - Original message -
> >>
> >> I realise this is a slightly different question (hence the new subject)
> >>
> >> OK, say I have an evil twin who wants to attack ('own') a lot of Nokia
> > N900
> >> devices. How do I do this?
> >
> > I hope that was retorical. Tell your evil twin to do something usefull.
>
> Err, no it wasn't retorical; it was hypothetical though in case you were
> worried.
>
> It's more about being responsible :)
> Actually it is very late in the day to be asking... but hey, it sounds like
> a
> topic worth raising.
>
> >> Does extras-testing factor into this?
> >
> > At least so that I would prefer maemo.org extras to be clean from
> > malware. It is much easier to promote it in Nokia internally when extras
> > contains good software.
>
> I agree 100% ... all it takes is one example of malware introduced into an
> OSS
> product and we (and Nokia) could lose a lot of credibility.
>
> I wonder how much that could be worth to some people? Maybe worth a
> deliberate
> attack? Maybe someone is playing a longer game?
>
> I just hope we are not planning on taking the "cross your fingers and toes
> *REALLY HARD* and hope everyone is nice to us" approach to security ;)
>
> Discuss...
>
> David
>
> --
> "Don't worry, you'll be fine; I saw it work in a cartoon once..."
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[David Greaves]

tero.k...@nokia.com wrote:
> - Original message -
>>
>> I realise this is a slightly different question (hence the new subject)
>>
>> OK, say I have an evil twin who wants to attack ('own') a lot of Nokia
> N900
>> devices. How do I do this?
> 
> I hope that was retorical. Tell your evil twin to do something usefull.

Err, no it wasn't retorical; it was hypothetical though in case you were 
worried.

It's more about being responsible :)
Actually it is very late in the day to be asking... but hey, it sounds like a
topic worth raising.

>> Does extras-testing factor into this?
> 
> At least so that I would prefer maemo.org extras to be clean from
> malware. It is much easier to promote it in Nokia internally when extras
> contains good software.

I agree 100% ... all it takes is one example of malware introduced into an OSS
product and we (and Nokia) could lose a lot of credibility.

I wonder how much that could be worth to some people? Maybe worth a deliberate
attack? Maybe someone is playing a longer game?

I just hope we are not planning on taking the "cross your fingers and toes
*REALLY HARD* and hope everyone is nice to us" approach to security ;)

Discuss...

David

-- 
"Don't worry, you'll be fine; I saw it work in a cartoon once..."
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: What's the best attack? (Re: How to use extras-testing correctly?)`

[tero.kojo]

- Original message -
>
> I realise this is a slightly different question (hence the new subject)
>
> OK, say I have an evil twin who wants to attack ('own') a lot of Nokia N900
> devices. How do I do this?

I hope that was retorical. Tell your evil twin to do something usefull.

> Does extras-testing factor into this?

At least so that I would prefer maemo.org extras to be clean from malware. It 
is much easier to promote it in Nokia internally when extras contains good 
software.

Tero

> David
>
> tero.k...@nokia.com wrote:
> > - Original message -
> > >
> > > On Thu, September 24, 2009 13:01, Aniello Del Sorbo wrote:
> > >
> > > > > > I am well aware of that :)
> > > > > > But if I go thru extras-testing (and I really want to!) then it
> > looks
> > > > > > like the Community has the last word on my application.
> > > > > >
> > > > > Yes, they do. It's a community effort, but look at it from the other
> > > > > side. Not one single person or entitiy can block your app. It
> > takes more
> > > > > people to block it.
> > > > >
> > > >
> > > > I know.. but still.. scares me.. :)
> > > >
> > > I tried to make this as transparent as possible, by showing each vote
> > > together with the user. If people are trolling we should be easily be
> > able
> > > to spot this.
> > >
> > > By letting the community doing this QA out in the open, we can prevent
> > > rejections without reasoning by a certain entity like we have seen in the
> > > news lately.
> >
> > This transparency is actually the thing that makes me feel secure about
> > the process. The testers are independent and operate with their own names.
> >
> > The (ex-)qa-manager in me is also excited by the fact that for once the
> > testers are really independant.
> >
> > > However, in a democracy not everybody can be satisfied. Let's tackle
> > > issues when we actually get there.
> >
> > Hear hear!
> > If the process does not work, then it get's changed. If it works we'll
> > just be happy and discuss how to make it more efficient.
> >
> > I'm already thinking that there might be a need for a Maemo testers'
> > club that makes sure that even niche apps don't get stranded in testing.
> >
> > Also I'll take the time to ask Nokia testing to look at the tooling
> > issue. I would like to have some nice set of tools for testing the
> > measurable aspects of applications (like battery usage as Igor pointed
> > out).
> >
> > And in any case we need to talk about Anidello's idea on feedback, with
> > beer or not.
> >
> > Tero
>
>
> --
> "Don't worry, you'll be fine; I saw it work in a cartoon once..."
>
>

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Quim Gil]

Vollmer Marius (Nokia-D/Helsinki) wrote:
> The only difference between Maemo Extras and such a new-world repository
> is that Maemo Extras is pre-configured when you buy your device from
> Nokia (but maybe not when you buy it from someone else), and the
> new-world repo needs to be added explicitly, which takes as much as one
> click on a .install file.

Another difference is that http://maemo.nokia.com/maemo-select/ and by
extension any promotion or support from Nokia will only pay attention to
 community software that exists in Extras or is clearly willing to get
there through the related QA process.

-- 
Quim Gil
open source advocate
Maemo Devices @ Nokia
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

What's the best attack? (Re: How to use extras-testing correctly?)`

[David Greaves]

I realise this is a slightly different question (hence the new subject)

OK, say I have an evil twin who wants to attack ('own') a lot of Nokia N900
devices. How do I do this?

Does extras-testing factor into this?

David

tero.k...@nokia.com wrote:
> - Original message -
>>
>> On Thu, September 24, 2009 13:01, Aniello Del Sorbo wrote:
>>
>> > > > I am well aware of that :)
>> > > > But if I go thru extras-testing (and I really want to!) then it
> looks
>> > > > like the Community has the last word on my application.
>> > > >
>> > > Yes, they do. It's a community effort, but look at it from the other
>> > > side. Not one single person or entitiy can block your app. It
> takes more
>> > > people to block it.
>> > >
>> >
>> > I know.. but still.. scares me.. :)
>> >
>> I tried to make this as transparent as possible, by showing each vote
>> together with the user. If people are trolling we should be easily be
> able
>> to spot this.
>>
>> By letting the community doing this QA out in the open, we can prevent
>> rejections without reasoning by a certain entity like we have seen in the
>> news lately.
> 
> This transparency is actually the thing that makes me feel secure about
> the process. The testers are independent and operate with their own names.
> 
> The (ex-)qa-manager in me is also excited by the fact that for once the
> testers are really independant.
> 
>> However, in a democracy not everybody can be satisfied. Let's tackle
>> issues when we actually get there.
> 
> Hear hear!
> If the process does not work, then it get's changed. If it works we'll
> just be happy and discuss how to make it more efficient.
> 
> I'm already thinking that there might be a need for a Maemo testers'
> club that makes sure that even niche apps don't get stranded in testing.
> 
> Also I'll take the time to ask Nokia testing to look at the tooling
> issue. I would like to have some nice set of tools for testing the
> measurable aspects of applications (like battery usage as Igor pointed
> out).
> 
> And in any case we need to talk about Anidello's idea on feedback, with
> beer or not.
> 
> Tero


-- 
"Don't worry, you'll be fine; I saw it work in a cartoon once..."
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Anderson Lizardo]

On Thu, Sep 24, 2009 at 1:43 PM,   wrote:
> Also I'll take the time to ask Nokia testing to look at the tooling issue. I
> would like to have some nice set of tools for testing the measurable aspects
> of applications (like battery usage as Igor pointed out).

For battery life analysis, maybe Nokia Energy Profiler for Maemo might
help. From this thread comment, it seems it will come with N900 too:

http://talk.maemo.org/showpost.php?p=324693&postcount=231

Regards,
-- 
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[tero.kojo]

- Original message -
>
> On Thu, September 24, 2009 13:01, Aniello Del Sorbo wrote:
>
> > > > I am well aware of that :)
> > > > But if I go thru extras-testing (and I really want to!) then it looks
> > > > like the Community has the last word on my application.
> > > >
> > > Yes, they do. It's a community effort, but look at it from the other
> > > side. Not one single person or entitiy can block your app. It takes more
> > > people to block it.
> > >
> >
> > I know.. but still.. scares me.. :)
> >
> I tried to make this as transparent as possible, by showing each vote
> together with the user. If people are trolling we should be easily be able
> to spot this.
>
> By letting the community doing this QA out in the open, we can prevent
> rejections without reasoning by a certain entity like we have seen in the
> news lately.

This transparency is actually the thing that makes me feel secure about the 
process. The testers are independent and operate with their own names.

The (ex-)qa-manager in me is also excited by the fact that for once the testers 
are really independant.

> However, in a democracy not everybody can be satisfied. Let's tackle
> issues when we actually get there.

Hear hear!
If the process does not work, then it get's changed. If it works we'll just be 
happy and discuss how to make it more efficient.

I'm already thinking that there might be a need for a Maemo testers' club that 
makes sure that even niche apps don't get stranded in testing.

Also I'll take the time to ask Nokia testing to look at the tooling issue. I 
would like to have some nice set of tools for testing the measurable aspects of 
applications (like battery usage as Igor pointed out).

And in any case we need to talk about Anidello's idea on feedback, with beer or 
not.

Tero

> >
> > --
> > anidel Sent from London, Eng, United Kingdom
> >
>
> - Niels
>
>
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
>

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Marius Vollmer]

ext Jeremiah Foster  writes:

>> x.The big fundamental difference between Debian and Maemo Extras is
>> that Debian is producing one big release of a complete integrated
>> distribution, while Maemo Extras is a collection of mostly
>> independent applications that are released independently.  Maemo
>> Extras is not collectively frozen at any point.  (It will cool down
>> as people lose interest in it, but nobody is waiting for a stable
>> release of Maemo Extras as a whole.)
>
> Well, I see them both as operating systems, so I don't think they are  
> really that different in keeping quality across the OS.

True, from the quality point of view, they have quite similar goals: all
the packages should work well together and form a consistent 'whole'.  

I was looking at releases: imagine that Nokia would develop platform
libraries in extras-devel as well, together with the applications, and
updates to the OS would be released by promoting them to extras (via
extras-testing), just like applications are released now.

With Debian's model, this would mean that applications in extras-devel
will be frozen and ultimately released together with the rest of the
platform.  I don't think this is acceptable.  Applications should move
to extras much more frequently than Nokia releases OS updates.

> The big difference between debian and Maemo is that debian is
> _completely_ free software while Maemo has closed bits.

Also, Debian is not developed behind closed doors.  But that's another
battle.
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Marius Vollmer]

"Stoppa Igor (Nokia-D/Helsinki)"  writes:

> It's not desire for control here, only for quality.
> Please remember that this thing still runs on battery.

And the iPhone runs on what?  Steve Jobs, err, bodily fluids? :)

Anyway, of course this is also about control: quality control.  The
difference with Apple is that there is no dictator built into the
maemo.org processes, or into the platform.

It will be a painful decision and will hopefully not be necessary, but
if Maemo Extras isn't doing the job that people expect from it, they can
rally together and 'fork' the whole thing by setting up their own
repositories.

The only difference between Maemo Extras and such a new-world repository
is that Maemo Extras is pre-configured when you buy your device from
Nokia (but maybe not when you buy it from someone else), and the
new-world repo needs to be added explicitly, which takes as much as one
click on a .install file.

But then again, Maemo Extras needs to be enabled explicitly, which might
be harder than clicking on a link after all.

> In the end it will be Nokia brand to get damaged if some app is
> available to the user for download and it literally sucks the battery
> dry without warning.

Users are warned a-plenty; now they even have to tick a checkbox to
confirm that the app they are about to install might harm the device.

> And, from user perspective, you do want to have someone to certify
> that your device will not be killed by the app.

Technically, packages in Maemo Extras are not 'certified'.  Those in
"Nokia Application" are certified, and you can install them without
being reminded that 90% of all software is crap.
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Jeremiah Foster]

On Sep 24, 2009, at 15:13, Marius Vollmer wrote:

> 2009/9/24 Jeremiah Foster :
>>
>> All community developed projects have a Quality Assurance process.  
>> Debian's,
>> which is what Maemo is based on, looks like this;
>>
>> 1. Submit your app to autobuilders
>> 2. If it fails to build from source, start over at 1
>> 3. If it builds, it stays in the new queue for ten days
>> 4. After ten days it automatically gets promoted to 'testing'
>> 5. After (roughly) eighteen months, testing is frozen (no more new  
>> apps)
>> 6. After all Release Critical bugs are fixed, testing becomes stable
>
> I think it is important to mention that Debian is much more careful  
> when
> giving upload rights to people.  It takes quite some dedication and  
> time
> (months) to even get to step 1 for the first time.

This is an important point - to upload to debian you need to be a  
Debian Developer, and that can easily take _years_. Maemo is open to  
anyone.
>
> Also, packages don't get promoted from the NEW queue to testing, they
> get promoted from unstable.  The very first upload of a package has to
> pass the NEW queue before it goes to unstable.  Subsequent uploads of
> the package go directly to unstable.  (This is just a detail.)

Actually, until unstable is 'frozen' into testing, they are the same  
thing.
>
> Also also, there are many more details to promotion than just waiting
> ten days, of course.  The salient point is that promotion happens by
> default unless it is stopped because of bugs, instead of being stopped
> by default unless there is enough karma to allow it.  (This is not  
> just
> a detail.)

This is something that we may want to think about - automatic  
promotion. I think stopping an app based on negative karma or test  
failures is fair, but having a criteria for promotion might mean good  
apps don't make it to users. This seems a little unfair.
>
>> Maemo is trying to innovate and crowd source the quality control and
>> shrink that time as much as possible.
>
> The big fundamental difference between Debian and Maemo Extras is that
> Debian is producing one big release of a complete integrated
> distribution, while Maemo Extras is a collection of mostly independent
> applications that are released independently.  Maemo Extras is not
> collectively frozen at any point.  (It will cool down as people lose
> interest in it, but nobody is waiting for a stable release of Maemo
> Extras as a whole.)

Well, I see them both as operating systems, so I don't think they are  
really that different in keeping quality across the OS. The big  
difference between debian and Maemo is that debian is _completely_  
free software while Maemo has closed bits.
>
> Maemo as a whole (including the platform produced by Nokia,  
> applications
> that are part of Nokia's releases such as Email and Sketch, and
> independently developed add-ons such as Maemo Extras) should innovate
> beyond Debian by defining a release process that allows multiple,
> largely independent 'modules' in the distribution that are released
> independently.

This is what Niels is trying to architect with our feedback.
>
> PS: "crowd source"?

http://en.wikipedia.org/wiki/Crowdsource

Jeremiah
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

RE: How to use extras-testing correctly?

[Igor.Stoppa]

From: maemo-developers-boun...@maemo.org [maemo-developers-boun...@maemo.org] 
On Behalf Of ext Aniello Del Sorbo [ani...@gmail.com]
Sent: 24 September 2009 12:19
To: Gil Quim (Nokia-D/Helsinki)
Cc: maemo-developers@maemo.org
Subject: Re: How to use extras-testing correctly?

I personally think we're taking a wrong approach to the problem.

I am not developing for  Apple because of their approval process and
their desire of control.

It's not desire for control here, only for quality.
Please remember that this thing still runs on battery.
In the end it will be Nokia brand to get damaged if some app
is available to the user for download and it literally sucks the battery dry 
without warning.

And, from user perspective, you do want to have someone to certify that your 
device will not be killed by the app.

cheers, igor
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Niels Breet]

On Thu, September 24, 2009 13:01, Aniello Del Sorbo wrote:

>>> I am well aware of that :)
>>> But if I go thru extras-testing (and I really want to!) then it looks
>>> like the Community has the last word on my application.
>>>
>> Yes, they do. It's a community effort, but look at it from the other
>> side. Not one single person or entitiy can block your app. It takes more
>> people to block it.
>>
>
> I know.. but still.. scares me.. :)
>
I tried to make this as transparent as possible, by showing each vote
together with the user. If people are trolling we should be easily be able
to spot this.

By letting the community doing this QA out in the open, we can prevent
rejections without reasoning by a certain entity like we have seen in the
news lately.

However, in a democracy not everybody can be satisfied. Let's tackle
issues when we actually get there.

>
> --
> anidel Sent from London, Eng, United Kingdom
>

- Niels


___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

I will bring the two bottles of beer and you will bring your free apologies. ;)

Aniello

2009/9/24 gary liquid :
> it will have to be on IOU
> i have no job, no money, no anything.
> im actually surprised mrs lcuk let me go at all
>
>
>
> On Thu, Sep 24, 2009 at 12:49 PM, Aniello Del Sorbo 
> wrote:
>>
>> Apologies will be accepted in Amsterdam along with a bottle of beer :D
>>
>> 2009/9/24 gary liquid :
>> > please accept huge apologies for spelling of your name Aniello :$
>> >
>> >
>> >
>> > On Thu, Sep 24, 2009 at 12:03 PM, gary liquid  wrote:
>> >>
>> >> :)
>> >>
>> >> this has been a good informative discussion.
>> >> since we are working on beta, can i suggest something.
>> >>
>> >> anidels point about app feedback *from the device itself* is important.
>> >>
>> >> if me as a user selects extras testing and installs a an app because
>> >> anidel said his new version  is there
>> >> from that moment on, it should be the device itself that monitors the
>> >> process.
>> >>
>> >> in the same way the crash reporter pops up and asks a question after a
>> >> crash, can we discuss a path to similar for testing apps?
>> >> perhaps if the app itself in testing had a flag added to its desktop
>> >> file
>> >> (or some other way, a list perhaps *hand waving*)
>> >> and when the user has run the app they are testing, or 1/2/12/24 hours
>> >> after installing the user was directly asked
>> >>
>> >> "what are your feelings on this app?"
>> >>
>> >> it would give control directly to the testers and remove the disconnect
>> >> that currently occurs - I for one have major problems maintaining a big
>> >> long
>> >> list of places and sites and things I should look for and the packages
>> >> list
>> >> overwhelms me with multiple versions and complex filenames which mean
>> >> nothing to me as a user.
>> >> the app is installed on my tablet, it is a computer, it knows the
>> >> version,
>> >> it knows the url path - i can give valuable feedback there and then.
>> >>
>> >> i dont know how practical or complex an app of this nature would be,
>> >> but
>> >> we have enough toolkits and developers around who might help code
>> >> something
>> >> up.
>> >> it should even go through extras-testing itself!
>> >>
>> >> gary
>> >>
>> >>
>> >>
>> >> On Thu, Sep 24, 2009 at 11:10 AM, Aniello Del Sorbo 
>> >> wrote:
>> >>>
>> >>> 2009/9/24 Andre Klapper :
>> >>> > Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del
>> >>> > Sorbo:
>> >>> >> My only complaint is on who has the final word.
>> >>> >
>> >>> > Again: It is the package maintainer, after his/her package has been
>> >>> > given 10 karma points by testers.
>> >>> >
>> >>> > andre
>> >>> > --
>> >>> > Andre Klapper (maemo.org bugmaster)
>> >>> >
>> >>>
>> >>> Thus I guess we need to work on what those testers should work.
>> >>> Is this being done?
>> >>>
>> >>> And what about the second part of the proposal?
>> >>>
>> >>> --
>> >>> anidel
>> >>> Sent from London, Eng, United Kingdom
>> >>> ___
>> >>> maemo-developers mailing list
>> >>> maemo-developers@maemo.org
>> >>> https://lists.maemo.org/mailman/listinfo/maemo-developers
>> >>
>> >
>> >
>>
>>
>>
>> --
>>
>> --
>> anidel
>
>



-- 

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Marius Vollmer]

2009/9/24 Jeremiah Foster :
>
> All community developed projects have a Quality Assurance process. Debian's,
> which is what Maemo is based on, looks like this;
>
> 1. Submit your app to autobuilders
> 2. If it fails to build from source, start over at 1
> 3. If it builds, it stays in the new queue for ten days
> 4. After ten days it automatically gets promoted to 'testing'
> 5. After (roughly) eighteen months, testing is frozen (no more new apps)
> 6. After all Release Critical bugs are fixed, testing becomes stable

I think it is important to mention that Debian is much more careful when
giving upload rights to people.  It takes quite some dedication and time
(months) to even get to step 1 for the first time.

Also, packages don't get promoted from the NEW queue to testing, they
get promoted from unstable.  The very first upload of a package has to
pass the NEW queue before it goes to unstable.  Subsequent uploads of
the package go directly to unstable.  (This is just a detail.)

Also also, there are many more details to promotion than just waiting
ten days, of course.  The salient point is that promotion happens by
default unless it is stopped because of bugs, instead of being stopped
by default unless there is enough karma to allow it.  (This is not just
a detail.)

> Maemo is trying to innovate and crowd source the quality control and
> shrink that time as much as possible.

The big fundamental difference between Debian and Maemo Extras is that
Debian is producing one big release of a complete integrated
distribution, while Maemo Extras is a collection of mostly independent
applications that are released independently.  Maemo Extras is not
collectively frozen at any point.  (It will cool down as people lose
interest in it, but nobody is waiting for a stable release of Maemo
Extras as a whole.)

Maemo as a whole (including the platform produced by Nokia, applications
that are part of Nokia's releases such as Email and Sketch, and
independently developed add-ons such as Maemo Extras) should innovate
beyond Debian by defining a release process that allows multiple,
largely independent 'modules' in the distribution that are released
independently.

I would like to hear announcements like "Modest 4.0 has entered Maemo
stable".  I would also like to hear this for Debian, actually, I don't
think its a feature that all of Debian freezes at the same time.

Some time ago, I tried to sketch a setup that might allow this (if it
works):

https://stage.maemo.org/svn/maemo/projects/haf/doc/mvo/system-model-2.txt

Now, what are the different flavours of Maemo Extras intended for?  I
would say:

 - extras-devel

   The development environment.  All new uploads are build in
   extras-devel.  Developers have extras-devel configured on their
   development boxes (in Scratchbox) and the buildbot installs build
   dependencies from it.

   Everybody subscribed to maemo-developers should have it configured on
   their devices (and should have a soldering iron ready to fix things
   when they break).

   Packages are allowed to enter extras-devel when they build and do not
   break the build environment.  It is OK if they can't be installed or
   make other packages uninstallable.

   Extras-devel is not for dangerous experiments, it is only for serious
   release candidates.  You should always be prepared that something
   that you upload to extras-devel reaches extras in a matter of days
   and will show up on end-user devices.

 - extras-testing

   Smoke testing.  Everybody who considers him/herself to be part of the
   Maemo community, everybody who attends the Maemo summit, everybody
   subscribed to maemo-users is expected to have extras-testing
   configured on their devices.

   As soon as you know that extras-testing exists, you should seriously
   consider using it.  You should tell your SO to use it.  Everyone who
   has a channel to contribute feedback should be invited.

   Packages can move to extras-testing when they are installable and no
   other package in extras-testing becomes uninstallable, among other
   criteria.

   It is important that also extras-devel can be used for beta testing
   as well, by developers.  As soon as something is in extras-devel,
   people should be able to give feedback about it in the same way that
   they would do it once the package is in extras-testing.

 - extras

   Our end users.  People who got conned into buying a two year
   subscription plan by an operator and who don't know the difference
   between S60 and Maemo, or for that matter, between Motorola and
   Nokia.


A drawback of this setup might be that you can only opt into being a
smoke tester for the whole of Extras, not for a specific application.  I
dont think this is actually a problem, since smoke testing is not beta
testing.

If you want to release a beta of something, we can try to hold it in
extras-testing so that end users are never tempted to install it.
However, sometimes you want to reach end user 

Re: How to use extras-testing correctly?

[gary liquid]

it will have to be on IOU
i have no job, no money, no anything.
im actually surprised mrs lcuk let me go at all



On Thu, Sep 24, 2009 at 12:49 PM, Aniello Del Sorbo wrote:

> Apologies will be accepted in Amsterdam along with a bottle of beer :D
>
> 2009/9/24 gary liquid :
> > please accept huge apologies for spelling of your name Aniello :$
> >
> >
> >
> > On Thu, Sep 24, 2009 at 12:03 PM, gary liquid  wrote:
> >>
> >> :)
> >>
> >> this has been a good informative discussion.
> >> since we are working on beta, can i suggest something.
> >>
> >> anidels point about app feedback *from the device itself* is important.
> >>
> >> if me as a user selects extras testing and installs a an app because
> >> anidel said his new version  is there
> >> from that moment on, it should be the device itself that monitors the
> >> process.
> >>
> >> in the same way the crash reporter pops up and asks a question after a
> >> crash, can we discuss a path to similar for testing apps?
> >> perhaps if the app itself in testing had a flag added to its desktop
> file
> >> (or some other way, a list perhaps *hand waving*)
> >> and when the user has run the app they are testing, or 1/2/12/24 hours
> >> after installing the user was directly asked
> >>
> >> "what are your feelings on this app?"
> >>
> >> it would give control directly to the testers and remove the disconnect
> >> that currently occurs - I for one have major problems maintaining a big
> long
> >> list of places and sites and things I should look for and the packages
> list
> >> overwhelms me with multiple versions and complex filenames which mean
> >> nothing to me as a user.
> >> the app is installed on my tablet, it is a computer, it knows the
> version,
> >> it knows the url path - i can give valuable feedback there and then.
> >>
> >> i dont know how practical or complex an app of this nature would be, but
> >> we have enough toolkits and developers around who might help code
> something
> >> up.
> >> it should even go through extras-testing itself!
> >>
> >> gary
> >>
> >>
> >>
> >> On Thu, Sep 24, 2009 at 11:10 AM, Aniello Del Sorbo 
> >> wrote:
> >>>
> >>> 2009/9/24 Andre Klapper :
> >>> > Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del Sorbo:
> >>> >> My only complaint is on who has the final word.
> >>> >
> >>> > Again: It is the package maintainer, after his/her package has been
> >>> > given 10 karma points by testers.
> >>> >
> >>> > andre
> >>> > --
> >>> > Andre Klapper (maemo.org bugmaster)
> >>> >
> >>>
> >>> Thus I guess we need to work on what those testers should work.
> >>> Is this being done?
> >>>
> >>> And what about the second part of the proposal?
> >>>
> >>> --
> >>> anidel
> >>> Sent from London, Eng, United Kingdom
> >>> ___
> >>> maemo-developers mailing list
> >>> maemo-developers@maemo.org
> >>> https://lists.maemo.org/mailman/listinfo/maemo-developers
> >>
> >
> >
>
>
>
> --
>
> --
> anidel
>
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

Apologies will be accepted in Amsterdam along with a bottle of beer :D

2009/9/24 gary liquid :
> please accept huge apologies for spelling of your name Aniello :$
>
>
>
> On Thu, Sep 24, 2009 at 12:03 PM, gary liquid  wrote:
>>
>> :)
>>
>> this has been a good informative discussion.
>> since we are working on beta, can i suggest something.
>>
>> anidels point about app feedback *from the device itself* is important.
>>
>> if me as a user selects extras testing and installs a an app because
>> anidel said his new version  is there
>> from that moment on, it should be the device itself that monitors the
>> process.
>>
>> in the same way the crash reporter pops up and asks a question after a
>> crash, can we discuss a path to similar for testing apps?
>> perhaps if the app itself in testing had a flag added to its desktop file
>> (or some other way, a list perhaps *hand waving*)
>> and when the user has run the app they are testing, or 1/2/12/24 hours
>> after installing the user was directly asked
>>
>> "what are your feelings on this app?"
>>
>> it would give control directly to the testers and remove the disconnect
>> that currently occurs - I for one have major problems maintaining a big long
>> list of places and sites and things I should look for and the packages list
>> overwhelms me with multiple versions and complex filenames which mean
>> nothing to me as a user.
>> the app is installed on my tablet, it is a computer, it knows the version,
>> it knows the url path - i can give valuable feedback there and then.
>>
>> i dont know how practical or complex an app of this nature would be, but
>> we have enough toolkits and developers around who might help code something
>> up.
>> it should even go through extras-testing itself!
>>
>> gary
>>
>>
>>
>> On Thu, Sep 24, 2009 at 11:10 AM, Aniello Del Sorbo 
>> wrote:
>>>
>>> 2009/9/24 Andre Klapper :
>>> > Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del Sorbo:
>>> >> My only complaint is on who has the final word.
>>> >
>>> > Again: It is the package maintainer, after his/her package has been
>>> > given 10 karma points by testers.
>>> >
>>> > andre
>>> > --
>>> > Andre Klapper (maemo.org bugmaster)
>>> >
>>>
>>> Thus I guess we need to work on what those testers should work.
>>> Is this being done?
>>>
>>> And what about the second part of the proposal?
>>>
>>> --
>>> anidel
>>> Sent from London, Eng, United Kingdom
>>> ___
>>> maemo-developers mailing list
>>> maemo-developers@maemo.org
>>> https://lists.maemo.org/mailman/listinfo/maemo-developers
>>
>
>



-- 

--
anidel
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

RE: How to use extras-testing correctly?

[Simon Pickering]

> you have installed an app from the -testing repository.
> you are therefore signing up to be a tester.
> 
> i dont expect normal apps to have this procedure.
> it is merely a way to expedite feedback 

Ah yes, forgot this was specific to -testing.


Simon


___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Matan Ziv-Av]

On Thu, Sep 24, 2009 at 3:25 PM, gary liquid  wrote:

> you have installed an app from the -testing repository.
> you are therefore signing up to be a tester.

Or a new version which includes a feature / bugfix you really want is
in quarantine.


-- 
Matan Ziv-Av.
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[gary liquid]

On Thu, Sep 24, 2009 at 12:22 PM, Simon Pickering
wrote:

>
> > this has been a good informative discussion.
> > since we are working on beta, can i suggest something.
> >
> > anidels point about app feedback *from the device itself* is
> > important.
> >
> > if me as a user selects extras testing and installs a an app
> > because anidel said his new version  is there from that
> > moment on, it should be the device itself that monitors the process.
> >
> > in the same way the crash reporter pops up and asks a
> > question after a crash, can we discuss a path to similar for
> > testing apps?
> > perhaps if the app itself in testing had a flag added to its
> > desktop file (or some other way, a list perhaps *hand
> > waving*) and when the user has run the app they are testing,
> > or 1/2/12/24 hours after installing the user was directly asked
> >
> > "what are your feelings on this app?"
>
> That would annoy me something rotten. I'd prefer an opt-in system, e.g. by
> adding a "Feedback/ratings" section to the application manager in which
> installed apps are listed and users can then rate/comment on them. That way
> if there's something especially good/bad I can go there and comment, but if
> I don't want to I don't have to.
>

you have installed an app from the -testing repository.
you are therefore signing up to be a tester.

i dont expect normal apps to have this procedure.
it is merely a way to expedite feedback

>
> However, I do like the idea of asking people why they are uninstalling
> something, just as long as it uses radio buttons/listbox for the responses
> (so it's quick and I don't have to type anything if I don't want to), with
> a
> text input box for extra comments/description.
>

yeah - crash feedback mechanism currently has similar
even windows server has this sort of thing if you try and reboot!

>
> Cheers,
>
>
> Simon
>
>
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

RE: How to use extras-testing correctly?

[Simon Pickering]

 
> this has been a good informative discussion.
> since we are working on beta, can i suggest something.
> 
> anidels point about app feedback *from the device itself* is 
> important.
> 
> if me as a user selects extras testing and installs a an app 
> because anidel said his new version  is there from that 
> moment on, it should be the device itself that monitors the process.
> 
> in the same way the crash reporter pops up and asks a 
> question after a crash, can we discuss a path to similar for 
> testing apps?
> perhaps if the app itself in testing had a flag added to its 
> desktop file (or some other way, a list perhaps *hand 
> waving*) and when the user has run the app they are testing, 
> or 1/2/12/24 hours after installing the user was directly asked
> 
> "what are your feelings on this app?"

That would annoy me something rotten. I'd prefer an opt-in system, e.g. by
adding a "Feedback/ratings" section to the application manager in which
installed apps are listed and users can then rate/comment on them. That way
if there's something especially good/bad I can go there and comment, but if
I don't want to I don't have to.

However, I do like the idea of asking people why they are uninstalling
something, just as long as it uses radio buttons/listbox for the responses
(so it's quick and I don't have to type anything if I don't want to), with a
text input box for extra comments/description.

Cheers,


Simon

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[gary liquid]

please accept huge apologies for spelling of your name Aniello :$



On Thu, Sep 24, 2009 at 12:03 PM, gary liquid  wrote:

> :)
>
> this has been a good informative discussion.
> since we are working on beta, can i suggest something.
>
> anidels point about app feedback *from the device itself* is important.
>
> if me as a user selects extras testing and installs a an app because anidel
> said his new version  is there
> from that moment on, it should be the device itself that monitors the
> process.
>
> in the same way the crash reporter pops up and asks a question after a
> crash, can we discuss a path to similar for testing apps?
> perhaps if the app itself in testing had a flag added to its desktop file
> (or some other way, a list perhaps *hand waving*)
> and when the user has run the app they are testing, or 1/2/12/24 hours
> after installing the user was directly asked
>
> "what are your feelings on this app?"
>
> it would give control directly to the testers and remove the disconnect
> that currently occurs - I for one have major problems maintaining a big long
> list of places and sites and things I should look for and the packages list
> overwhelms me with multiple versions and complex filenames which mean
> nothing to me as a user.
> the app is installed on my tablet, it is a computer, it knows the version,
> it knows the url path - i can give valuable feedback there and then.
>
> i dont know how practical or complex an app of this nature would be, but we
> have enough toolkits and developers around who might help code something up.
> it should even go through extras-testing itself!
>
> gary
>
>
>
>
> On Thu, Sep 24, 2009 at 11:10 AM, Aniello Del Sorbo wrote:
>
>> 2009/9/24 Andre Klapper :
>> > Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del Sorbo:
>> >> My only complaint is on who has the final word.
>> >
>> > Again: It is the package maintainer, after his/her package has been
>> > given 10 karma points by testers.
>> >
>> > andre
>> > --
>> > Andre Klapper (maemo.org bugmaster)
>> >
>>
>> Thus I guess we need to work on what those testers should work.
>> Is this being done?
>>
>> And what about the second part of the proposal?
>>
>> --
>> anidel
>> Sent from London, Eng, United Kingdom
>> ___
>> maemo-developers mailing list
>> maemo-developers@maemo.org
>> https://lists.maemo.org/mailman/listinfo/maemo-developers
>>
>
>
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[gary liquid]

:)

this has been a good informative discussion.
since we are working on beta, can i suggest something.

anidels point about app feedback *from the device itself* is important.

if me as a user selects extras testing and installs a an app because anidel
said his new version  is there
from that moment on, it should be the device itself that monitors the
process.

in the same way the crash reporter pops up and asks a question after a
crash, can we discuss a path to similar for testing apps?
perhaps if the app itself in testing had a flag added to its desktop file
(or some other way, a list perhaps *hand waving*)
and when the user has run the app they are testing, or 1/2/12/24 hours after
installing the user was directly asked

"what are your feelings on this app?"

it would give control directly to the testers and remove the disconnect that
currently occurs - I for one have major problems maintaining a big long list
of places and sites and things I should look for and the packages list
overwhelms me with multiple versions and complex filenames which mean
nothing to me as a user.
the app is installed on my tablet, it is a computer, it knows the version,
it knows the url path - i can give valuable feedback there and then.

i dont know how practical or complex an app of this nature would be, but we
have enough toolkits and developers around who might help code something up.
it should even go through extras-testing itself!

gary



On Thu, Sep 24, 2009 at 11:10 AM, Aniello Del Sorbo wrote:

> 2009/9/24 Andre Klapper :
> > Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del Sorbo:
> >> My only complaint is on who has the final word.
> >
> > Again: It is the package maintainer, after his/her package has been
> > given 10 karma points by testers.
> >
> > andre
> > --
> > Andre Klapper (maemo.org bugmaster)
> >
>
> Thus I guess we need to work on what those testers should work.
> Is this being done?
>
> And what about the second part of the proposal?
>
> --
> anidel
> Sent from London, Eng, United Kingdom
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

2009/9/24 Andre Klapper :
> Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del Sorbo:
>> My only complaint is on who has the final word.
>
> Again: It is the package maintainer, after his/her package has been
> given 10 karma points by testers.
>
> andre
> --
> Andre Klapper (maemo.org bugmaster)
>

Thus I guess we need to work on what those testers should work.
Is this being done?

And what about the second part of the proposal?

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

2009/9/24 Jeremiah Foster :
>
> On Sep 24, 2009, at 12:48, Aniello Del Sorbo wrote:
>
>> 2009/9/24 Quim Gil :
>>>
>>> ext Aniello Del Sorbo wrote:

 I am not developing for  Apple because of their approval process and
 their desire of control.
>>>
>>> What is your opinion about Deboan, Ubuntu, Fedora... all of them having
>>> also QA processes in place in order to make it to their stable releases?
>>>
>> I don't know them.
>
> All community developed projects have a Quality Assurance process. Debian's,
> which is what Maemo is based on, looks like this;
>
> 1. Submit your app to autobuilders
> 2. If it fails to build from source, start over at 1
> 3. If it builds, it stays in the new queue for ten days
> 4. After ten days it automatically gets promoted to 'testing'
> 5. After (roughly) eighteen months, testing is frozen (no more new apps)
> 6. After all Release Critical bugs are fixed, testing becomes stable
>
> Maemo is trying to innovate and crowd source the quality control and shrink
> that time as much as possible.
>

Uhm...
I see the point.
If I want my new application to be in the Official Debian repository,
it goes thru all this testing.
It makes sense.

>> But I understand it's a different matter and I think I am missing the
>> point of the whole Extras Testing infrastructure.
>> I see it as a very useful tool to help bringing good quality
>> application in the hands of people.
>> I like it.
>> My only complaint is on who has the final word.
>
> Your software lives in an ecosystem of autobuilders, repositories, and end
> user tablets. It has to co-operate with all these resources to be of value
> to a user. You have a responsibility to not wipe out a user's hard drive,
> etc.
>

I know and I do take that responsibility into account.

> I understand your interest in getting your software into your user's hands
> faster, but the QA process will help with this. If we can build the
> reputation of packages in Extras being high-quality, users will be more
> willing to download them. We have to build that reputation first.
>

No I don't want my app to go faster in the user's hand.
I want it to reach that hand when it's ready.

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Jeremiah Foster]

On Sep 24, 2009, at 12:48, Aniello Del Sorbo wrote:

> 2009/9/24 Quim Gil :
>>
>> ext Aniello Del Sorbo wrote:
>>> I am not developing for  Apple because of their approval process and
>>> their desire of control.
>>
>> What is your opinion about Deboan, Ubuntu, Fedora... all of them  
>> having
>> also QA processes in place in order to make it to their stable  
>> releases?
>>
> I don't know them.

All community developed projects have a Quality Assurance process.  
Debian's, which is what Maemo is based on, looks like this;

1. Submit your app to autobuilders
2. If it fails to build from source, start over at 1
3. If it builds, it stays in the new queue for ten days
4. After ten days it automatically gets promoted to 'testing'
5. After (roughly) eighteen months, testing is frozen (no more new apps)
6. After all Release Critical bugs are fixed, testing becomes stable

Maemo is trying to innovate and crowd source the quality control and  
shrink that time as much as possible.

> But I understand it's a different matter and I think I am missing the
> point of the whole Extras Testing infrastructure.
> I see it as a very useful tool to help bringing good quality
> application in the hands of people.
> I like it.
> My only complaint is on who has the final word.

Your software lives in an ecosystem of autobuilders, repositories, and  
end user tablets. It has to co-operate with all these resources to be  
of value to a user. You have a responsibility to not wipe out a user's  
hard drive, etc.

I understand your interest in getting your software into your user's  
hands faster, but the QA process will help with this. If we can build  
the reputation of packages in Extras being high-quality, users will be  
more willing to download them. We have to build that reputation first.

Jeremiah


___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

2009/9/24 Niels Breet :
> On Thu, September 24, 2009 12:25, Aniello Del Sorbo wrote:
>> 2009/9/24  :
>>
>>>> -Original Message-
>>>> From: maemo-developers-boun...@maemo.org
>>>> [mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext
>>>> Aniello Del Sorbo
>>>> Sent: 24 September, 2009 12:20
>>>> To: Gil Quim (Nokia-D/Helsinki)
>>>> Cc: maemo-developers@maemo.org
>>>> Subject: Re: How to use extras-testing correctly?
>>>>
>>>>
>>>> I personally think we're taking a wrong approach to the problem.
>>>>
>>>>
>>>> I am not developing for  Apple because of their approval
>>>> process and their desire of control.
>>>
>>> That is a different matter alltogether. One device, one shop, complete
>>> control. That has very little to do with a community site like
>>> maemo.org and a deevice using debian repositories.
>>>
>>
>> I am well aware of that :)
>> But if I go thru extras-testing (and I really want to!) then it looks
>> like the Community has the last word on my application.
>>
> Yes, they do. It's a community effort, but look at it from the other side.
> Not one single person or entitiy can block your app. It takes more people
> to block it.
>

I know.. but still.. scares me.. :)

>>>> I don't want my application to go through an approval process
>>>> to reach my users.
>>>
>>> No one is forcing you to place your app in extras. However it is the
>>> place most likely to be found by new users. Maemo Extras should be of
>>> high quality because we will have completely new users to the platform.
>>> We don't want to scare them away, simple as that.
>>>
>>>
>>
>> And that's what I want. But I am the one that decides if my
>> application has to go from Extras Testing to Extras. If I decide badly, the
>> Users will uninstall my app and file bug via AM
>> and eventually it'll be pulled out from Extras. Only high quality apps
>> remain.
>>
>
> This really is the wrong way around. Let's say your app corrupted the
> device, then joe user really would have liked that this was caught before
> he ruined his device with it.
>
> Testing takes a bit of time, but has a lot of advantages in the long run.
>

I don't want that either, but I am sure if an app goes thru testing,
it's because the developer doesn't want that either.
And Testing DOES NOT guarantee this anyway.
But helps, but the developer has to collaborate.

I probably trust them more than you :)

>> If I don't get those, may be because few people care, I am stuck in
>> Testing.
>> And this reminds me of Apple approval process.
>>
> You are really looking at it the wrong way. There are only a few dozen
> people with devices now, of course it will take longer to test your app or
> gather a large enough crowd.
>
> I'm pretty sure that when there are thousands of people with devices, this
> will go a lot faster.
>

It will for sure. Well, I still have the feeling that Testing should not impose.
I will put my app thru it, and probably I won't have any issue in the end...
But can we then make something similar to the Testing infrastructure
available to Extras Devel as well?

> Anyway, what's the rush?
>

Who's rushing ? :) :p

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Andre Klapper]

Am Donnerstag, den 24.09.2009, 11:48 +0100 schrieb Aniello Del Sorbo:
> My only complaint is on who has the final word.

Again: It is the package maintainer, after his/her package has been
given 10 karma points by testers.

andre
-- 
Andre Klapper (maemo.org bugmaster)

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

2009/9/24 Niels Breet :
> On Thu, September 24, 2009 12:07, Henrik Hedberg wrote:
>> Niels Breet wrote:
>>
>>
>>> On Thu, September 24, 2009 11:19, Aniello Del Sorbo wrote:
>>>
 Here's my proposal:



 We leave Extras-Testing as it is, with votes, comments, bug reports
 an so on (or improve it, as you wish). But then it's only the
 developer that decides, based on that feedback, if or not to promote
 the application to Extras.


>>> This is what happens now. Even if your app gets enough votes for
>>> promotion to Extras, you still need to push the button. There is no
>>> automatic promotion.
>>>
>>> The developer/maintainer is always the one who decides on when to
>>> publish the app.
>>
>> So no quarantine, no need to gather karma points? Please, update the
>> wiki [1], if that will be the case.
>
> No, but the developer can decide to leave the app in extras-testing.
>
>>
>> Aniello was saying that developer should have an option to promote
>> his application directly into extras regardless of the extras-testing QA. I
>> second to that.
>>
>
> This is what we had in diablo and this clearly didn't work.
>
In Diablo we hadn't any feedback from extras-devel.
There was no infrastructure like Extras Testing there.

> The whole idea about this QA testing is to prevent the wild west
> brokenness we had before. Sure it is not perfect, we can shape it
> gradually.
>

That's why I love it.

> Remember that this is still beta, nothing is perfect in the first release.
>

That's exactly what I am doing.

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Niels Breet]

On Thu, September 24, 2009 12:25, Aniello Del Sorbo wrote:
> 2009/9/24  :
>
>>> -Original Message-
>>> From: maemo-developers-boun...@maemo.org
>>> [mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext
>>> Aniello Del Sorbo
>>> Sent: 24 September, 2009 12:20
>>> To: Gil Quim (Nokia-D/Helsinki)
>>> Cc: maemo-developers@maemo.org
>>> Subject: Re: How to use extras-testing correctly?
>>>
>>>
>>> I personally think we're taking a wrong approach to the problem.
>>>
>>>
>>> I am not developing for  Apple because of their approval
>>> process and their desire of control.
>>
>> That is a different matter alltogether. One device, one shop, complete
>> control. That has very little to do with a community site like
>> maemo.org and a deevice using debian repositories.
>>
>
> I am well aware of that :)
> But if I go thru extras-testing (and I really want to!) then it looks
> like the Community has the last word on my application.
>
Yes, they do. It's a community effort, but look at it from the other side.
Not one single person or entitiy can block your app. It takes more people
to block it.

>>> I don't want my application to go through an approval process
>>> to reach my users.
>>
>> No one is forcing you to place your app in extras. However it is the
>> place most likely to be found by new users. Maemo Extras should be of
>> high quality because we will have completely new users to the platform.
>> We don't want to scare them away, simple as that.
>>
>>
>
> And that's what I want. But I am the one that decides if my
> application has to go from Extras Testing to Extras. If I decide badly, the
> Users will uninstall my app and file bug via AM
> and eventually it'll be pulled out from Extras. Only high quality apps
> remain.
>

This really is the wrong way around. Let's say your app corrupted the
device, then joe user really would have liked that this was caught before
he ruined his device with it.

Testing takes a bit of time, but has a lot of advantages in the long run.


>
> If I don't get those, may be because few people care, I am stuck in
> Testing.
> And this reminds me of Apple approval process.
>
You are really looking at it the wrong way. There are only a few dozen
people with devices now, of course it will take longer to test your app or
gather a large enough crowd.

I'm pretty sure that when there are thousands of people with devices, this
will go a lot faster.

Anyway, what's the rush?

>
>
> Aniello

- Niels

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

No wait...

The line is a bit blurry.

My proposal was based on the assumption that developers are working on
their application for the good of the whole Maemo community.
They have a moral duty of approving only good quality software and
Testing is a good tool to achieve that.

I don't want developers to approve their applications just because it
was in Testing for ages and few people voted it or because it was
badly voted or because of a bug that's not in his application.

If he has the last word, fine, just push in him this moral duty.
If he hasn't control, then who decides the application future in those
cases and how/when?

Aniello

2009/9/24  :
>> -Original Message-
>> From: maemo-developers-boun...@maemo.org
>> [mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext
>> Henrik Hedberg
>> Sent: 24 September, 2009 13:07
>> Cc: maemo-developers@maemo.org
>> Subject: Re: How to use extras-testing correctly?
>>
>> Niels Breet wrote:
>> > The developer/maintainer is always the one who decides on when to
>> > publish the app.
>>
>>     So no quarantine, no need to gather karma points? Please,
>> update the wiki [1], if that will be the case.
>
> No, the question was who makes the promotion to extras. The developer does in 
> the end.
>
>>     Aniello was saying that developer should have an option
>> to promote his application directly into extras regardless of
>> the extras-testing QA. I second to that.
>
> I find that understandable and scary. You guys want the control on your 
> software, naturally.
>
> But giving a potentially bad experience to a first time oss user? No. The 
> people who will get N900s aren't all reading this list, and some of them are 
> even a bit clueless. I know that the people who are raising this issue 
> provide a class software, I use it. But not everything in for instance Diablo 
> extras is good and functional.
>
> Even worse, direct promotion leaves a gaping hole to upload stuff that 
> crashes devices. Which is then pulled out later?
> I know that the people looking after the maemo.org repositories are fast, but 
> even they go on vacations occasionally.
>
>>     In addition, Aniello had nice ideas about the processes
>> and tools how to find applications (and developers) who had
>> probably misused their power to promote application directly
>> into extras. There is no hurry to implement those, although
>> they could make sense in the long run.
>
> As a feedback channel yes. It's a good idea, but needs an implementation, 
> which we could talk about in Amsterdam (a lot of the early contributors will 
> be there, I don't mean to discriminate) live and on a thread here.
>
> Tero
>
>>     BR,
>>
>>     Henrik
>>
>> [1] http://wiki.maemo.org/Extras-testing#Promotion_.2F_Demotion
>>
>> --
>>     Henrik Hedberg  -  http://www.henrikhedberg.net/
>> ___
>> maemo-developers mailing list
>> maemo-developers@maemo.org
>> https://lists.maemo.org/mailman/listinfo/maemo-developers
>>
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>



-- 

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

2009/9/24 Quim Gil :
>
> ext Aniello Del Sorbo wrote:
>> I am not developing for  Apple because of their approval process and
>> their desire of control.
>
> What is your opinion about Deboan, Ubuntu, Fedora... all of them having
> also QA processes in place in order to make it to their stable releases?
>

I don't know them.
But I understand it's a different matter and I think I am missing the
point of the whole Extras Testing infrastructure.
I see it as a very useful tool to help bringing good quality
application in the hands of people.
I like it.
My only complaint is on who has the final word.

>> I don't want my application to go through an approval process to reach my 
>> users.
>> I think that the developer HAS the last work on whether or not her
>> application is ready to be promoted to Extras.
>
> Your work as an individual developer has an impact on the overall image
> the Maemo platform and even the Maemo devices have in the hands of real
> users. maemo.org wants to keep good quality for community applications
> and this is why this community qa process was started.
>
> You can still skip it completely by convicing your users to install the
> version in extras devel, go ahead with your own repo... From a community
> point of view you are simply not encouraged to do so and follow the QA
> process.
>

I don't want to set up my own repo, but I wanted to make good use of
this tool (extras testing)
I have this complaint on who has the last word on the application
going to extras.
It's still not clear. Neils stated we do have such control.
Is that true, then?

>
>> But I like the idea of my application being tested and I'd love to get
>> feedback from the final users.
>> Android has a nice approach, I think the application goes straight to
>> Market and when a user removes it, she's being asked why.
>
> After going through
> http://www.android.com/us/developer-distribution-agreement.html - which
> I reckon I haven't read entirely.
>

Me neither. But it doesn't matter, I liked the part when it asks me
why I am removing the application.

>> I learned that the extremes are almost never the best choice.
>> Thus, I would like to see is a mix of Apple idea of "testers" and
>> Google Android "uninstallation survey".
>>
>> Here's my proposal:
>>
>> We leave Extras-Testing as it is, with votes, comments, bug reports an
>> so on (or improve it, as you wish).
>> But then it's only the developer that decides, based on that feedback,
>> if or not to promote the application to Extras.
>>
>> When the application gets promoted to Extras, people will start using it.
>>
>> If there's a major bug, the user will uninstall the application.
>>
>> The Application Manager should ask the user, as Android Market does,
>> WHY she uninstalling the application.
>> Of course she may choose not to give any feedback.
>> One of the choice she has is "Uninstalling because of a bug" and she
>> can fill in the bug if she wish so.
>
> Implementing this in the Application Manager before the Maemo 5 final
> release is impossible. Only debating it here might take longer.
>

I am not pushing this idea so that i can be implemented now.
But I feel it can help keeping up the quality of applications in Extras.
Independently from my first point.

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Niels Breet]

On Thu, September 24, 2009 12:07, Henrik Hedberg wrote:
> Niels Breet wrote:
>
>
>> On Thu, September 24, 2009 11:19, Aniello Del Sorbo wrote:
>>
>>> Here's my proposal:
>>>
>>>
>>>
>>> We leave Extras-Testing as it is, with votes, comments, bug reports
>>> an so on (or improve it, as you wish). But then it's only the
>>> developer that decides, based on that feedback, if or not to promote
>>> the application to Extras.
>>>
>>>
>> This is what happens now. Even if your app gets enough votes for
>> promotion to Extras, you still need to push the button. There is no
>> automatic promotion.
>>
>> The developer/maintainer is always the one who decides on when to
>> publish the app.
>
> So no quarantine, no need to gather karma points? Please, update the
> wiki [1], if that will be the case.

No, but the developer can decide to leave the app in extras-testing.

>
> Aniello was saying that developer should have an option to promote
> his application directly into extras regardless of the extras-testing QA. I
> second to that.
>

This is what we had in diablo and this clearly didn't work.

The whole idea about this QA testing is to prevent the wild west
brokenness we had before. Sure it is not perfect, we can shape it
gradually.

Remember that this is still beta, nothing is perfect in the first release.

> BR,
>
>
> Henrik
>
>
> [1] http://wiki.maemo.org/Extras-testing#Promotion_.2F_Demotion
>
>
> --
> Henrik Hedberg  -  http://www.henrikhedberg.net/

- Niels

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

RE: How to use extras-testing correctly?

[tero.kojo]

> -Original Message-
> From: maemo-developers-boun...@maemo.org 
> [mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext 
> Henrik Hedberg
> Sent: 24 September, 2009 13:07
> Cc: maemo-developers@maemo.org
> Subject: Re: How to use extras-testing correctly?
> 
> Niels Breet wrote:
> > The developer/maintainer is always the one who decides on when to 
> > publish the app.
> 
> So no quarantine, no need to gather karma points? Please, 
> update the wiki [1], if that will be the case.

No, the question was who makes the promotion to extras. The developer does in 
the end.

> Aniello was saying that developer should have an option 
> to promote his application directly into extras regardless of 
> the extras-testing QA. I second to that.

I find that understandable and scary. You guys want the control on your 
software, naturally.

But giving a potentially bad experience to a first time oss user? No. The 
people who will get N900s aren't all reading this list, and some of them are 
even a bit clueless. I know that the people who are raising this issue provide 
a class software, I use it. But not everything in for instance Diablo extras is 
good and functional.

Even worse, direct promotion leaves a gaping hole to upload stuff that crashes 
devices. Which is then pulled out later?
I know that the people looking after the maemo.org repositories are fast, but 
even they go on vacations occasionally.

> In addition, Aniello had nice ideas about the processes 
> and tools how to find applications (and developers) who had 
> probably misused their power to promote application directly 
> into extras. There is no hurry to implement those, although 
> they could make sense in the long run.

As a feedback channel yes. It's a good idea, but needs an implementation, which 
we could talk about in Amsterdam (a lot of the early contributors will be 
there, I don't mean to discriminate) live and on a thread here.

Tero

> BR,
> 
> Henrik
> 
> [1] http://wiki.maemo.org/Extras-testing#Promotion_.2F_Demotion
> 
> -- 
> Henrik Hedberg  -  http://www.henrikhedberg.net/ 
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
> 
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

2009/9/24  :
>> -Original Message-
>> From: maemo-developers-boun...@maemo.org
>> [mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext
>> Aniello Del Sorbo
>> Sent: 24 September, 2009 12:20
>> To: Gil Quim (Nokia-D/Helsinki)
>> Cc: maemo-developers@maemo.org
>> Subject: Re: How to use extras-testing correctly?
>>
>> I personally think we're taking a wrong approach to the problem.
>>
>> I am not developing for  Apple because of their approval
>> process and their desire of control.
>
> That is a different matter alltogether. One device, one shop, complete 
> control. That has very little to do with a community site like maemo.org and 
> a deevice using debian repositories.
>

I am well aware of that :)
But if I go thru extras-testing (and I really want to!) then it looks
like the Community has the last word on my application.

>> I don't want my application to go through an approval process
>> to reach my users.
>
> No one is forcing you to place your app in extras. However it is the place 
> most likely to be found by new users.
> Maemo Extras should be of high quality because we will have completely new 
> users to the platform. We don't want to scare them away, simple as that.
>

And that's what I want. But I am the one that decides if my
application has to go from Extras Testing to Extras.
If I decide badly, the Users will uninstall my app and file bug via AM
and eventually it'll be pulled out from Extras.
Only high quality apps remain.

>> I think that the developer HAS the last work on whether or
>> not her application is ready to be promoted to Extras.
>
> The developer has the last word; just don't promote from testing.
>

But I do want to go through testing. That's why I am proposing this.

>> But I like the idea of my application being tested and I'd
>> love to get feedback from the final users.
>> Android has a nice approach, I think the application goes
>> straight to Market and when a user removes it, she's being asked why.
>
> I don't think so, please provide a reference.
>
I don't have any. I do know that if you uninstall an app, it asks you why.
I have an Android phone.
I know that.

>> I learned that the extremes are almost never the best choice.
>> Thus, I would like to see is a mix of Apple idea of "testers"
>> and Google Android "uninstallation survey".
>>
>> Here's my proposal:
>>
>> We leave Extras-Testing as it is, with votes, comments, bug
>> reports an so on (or improve it, as you wish).
>> But then it's only the developer that decides, based on that
>> feedback, if or not to promote the application to Extras.
>
> Which is how it works, except you need ten thumbs up.
>

Exactly what I am trying to remove.
I choose.
But based on those 9 thumbs up.
I need to be wise of course, but I choose.

>> When the application gets promoted to Extras, people will
>> start using it.
>
> Yes, extras-testing for testing, -devel for development, extras for prime 
> time.
>

/me nods

>> If there's a major bug, the user will uninstall the application.
>
> Certainly.
>

/me nods

>> The Application Manager should ask the user, as Android
>> Market does, WHY she uninstalling the application.
>> Of course she may choose not to give any feedback.
>> One of the choice she has is "Uninstalling because of a bug"
>> and she can fill in the bug if she wish so.
>>
>> This bug goes straight to bugzilla for that application and
>> we should count them and decide on two limits: a soft
>> threshold and a hard one.
>>
>> Soft threshold:
>> if the application goes below this soft limit a yellow flag
>> is raised (how to handle it, it's another topic).
>>
>> Hard threshold:
>> if the application goes below this hard limit a red flag is
>> raised and application pulled from Extras.
>
> Look, if you want to remove stuff, it's better to do it early than late. The 
> assumption is that someone will submit bad code, then the better option is to 
> take it out before it hits peoples devices. Not after it's gotten to the 
> public.
>

My point is exactly this. I think it's the opposite.
The decide what my application does, the users decide if it is good.
No one in between.
Extras Testing IS very good for.. testing.

>> This way the Developer has full control of when and how she
>> wants her application to reach the public.
>> The Community has some control of which applications should
>> be removed.
>
> Which is there already. When you get

Re: How to use extras-testing correctly?

[Quim Gil]

ext Aniello Del Sorbo wrote:
> I am not developing for  Apple because of their approval process and
> their desire of control.

What is your opinion about Deboan, Ubuntu, Fedora... all of them having
also QA processes in place in order to make it to their stable releases?


> I don't want my application to go through an approval process to reach my 
> users.
> I think that the developer HAS the last work on whether or not her
> application is ready to be promoted to Extras.

Your work as an individual developer has an impact on the overall image
the Maemo platform and even the Maemo devices have in the hands of real
users. maemo.org wants to keep good quality for community applications
and this is why this community qa process was started.

You can still skip it completely by convicing your users to install the
version in extras devel, go ahead with your own repo... From a community
point of view you are simply not encouraged to do so and follow the QA
process.


> But I like the idea of my application being tested and I'd love to get
> feedback from the final users.
> Android has a nice approach, I think the application goes straight to
> Market and when a user removes it, she's being asked why.

After going through
http://www.android.com/us/developer-distribution-agreement.html - which
I reckon I haven't read entirely.


> I learned that the extremes are almost never the best choice.
> Thus, I would like to see is a mix of Apple idea of "testers" and
> Google Android "uninstallation survey".
> 
> Here's my proposal:
> 
> We leave Extras-Testing as it is, with votes, comments, bug reports an
> so on (or improve it, as you wish).
> But then it's only the developer that decides, based on that feedback,
> if or not to promote the application to Extras.
> 
> When the application gets promoted to Extras, people will start using it.
> 
> If there's a major bug, the user will uninstall the application.
> 
> The Application Manager should ask the user, as Android Market does,
> WHY she uninstalling the application.
> Of course she may choose not to give any feedback.
> One of the choice she has is "Uninstalling because of a bug" and she
> can fill in the bug if she wish so.

Implementing this in the Application Manager before the Maemo 5 final
release is impossible. Only debating it here might take longer.

> 
> This bug goes straight to bugzilla for that application and we should
> count them and decide on two limits: a soft threshold and a hard one.
> 
> Soft threshold:
> if the application goes below this soft limit a yellow flag is raised
> (how to handle it, it's another topic).
> 
> Hard threshold:
> if the application goes below this hard limit a red flag is raised and
> application pulled from Extras.
> 
> This way the Developer has full control of when and how she wants her
> application to reach the public.
> The Community has some control of which applications should be removed.
> 
> A good application with a good developer stays there.
> A good application with a slow developer might get a yellow flag and
> developer may feel pushed more to fix bugs
> A good application with a bad developer might become buggy over time,
> outdated and gets pulled out
> A bad application with a good developer has a better chance to become a good 
> one
> A bad application with a slow developer has its life on the edge of a
> razor.. be ware
> A bad application with a bad developer is dead and not in Extras.
> 
> Aniello
> 
> 2009/9/24 Quim Gil :
>>
>> ext Fred Lefévère-Laoide wrote:
>>> I know you can't force people to vote or test ... But it means that if
>>> an app has a very limited public it doesn't get out of extras-testing ?
>> This is a very good point. I believe it won't be an issue once there are
>> more people with devices able to vote, but still.
>>
>> Something that would help is to have a strict chronological sorting at
>> http://maemo.org/packages/repository/qa/fremantle_extras-testing_free_armel/
>> so oldest submissions are always on top. And encourage people to help
>> kicking the apps on the top even if they don't use it normally. At the
>> end it's not that hard to give an app a shot and look whether the basic
>> QA factors are in place.
>>
>> I believe after the N900 sales start there will be also a sustainability
>> approach for the applications not used by many.
>>
>> Some will be highly specialized with a small but committed audience that
>> will help testing and ranking. If a bug goes through they will be the
>> ones suffering it first.  :)
>>
>> Some will be just starting and have a real small audience. But well, in
>> such cases a bug going through won't cause much harm either. In most
>> cases as the audience grows the less obvious bugs will pop up and then
>> the developer will need to respond with newer and better versions. If
>> major bugs are standing without fixes and the users of the app increase,
>> then the relevance of the bug will increase as well and it might imply
>> at some poi

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

I haven't got Neils e-mail yet.
But it looked like that the app gets promoted automatically or stay in
quarantine until it reaches enough votes.

If it stays in quarantine, I won't go for Extras Testing and I'll miss
an opportunity and I propose to remove this limitation.

If I have control of the approval process, I'll go thru it and wait
for no showstoppers before promoting my application.
But my second proposal to keep track of how the appication is going, stands.

Aniello

2009/9/24 Henrik Hedberg :
> Niels Breet wrote:
>
>> On Thu, September 24, 2009 11:19, Aniello Del Sorbo wrote:
>>> Here's my proposal:
>>>
>>>
>>> We leave Extras-Testing as it is, with votes, comments, bug reports an
>>> so on (or improve it, as you wish). But then it's only the developer that
>>> decides, based on that feedback, if or not to promote the application to
>>> Extras.
>>>
>> This is what happens now. Even if your app gets enough votes for promotion
>> to Extras, you still need to push the button. There is no automatic
>> promotion.
>>
>> The developer/maintainer is always the one who decides on when to publish
>> the app.
>
>    So no quarantine, no need to gather karma points? Please, update the
> wiki [1], if that will be the case.
>
>    Aniello was saying that developer should have an option to promote
> his application directly into extras regardless of the extras-testing
> QA. I second to that.
>
>    In addition, Aniello had nice ideas about the processes and tools
> how to find applications (and developers) who had probably misused their
> power to promote application directly into extras. There is no hurry to
> implement those, although they could make sense in the long run.
>
>    BR,
>
>    Henrik
>
> [1] http://wiki.maemo.org/Extras-testing#Promotion_.2F_Demotion
>
> --
>    Henrik Hedberg  -  http://www.henrikhedberg.net/
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>



-- 

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

RE: How to use extras-testing correctly?

[tero.kojo]

> -Original Message-
> From: maemo-developers-boun...@maemo.org 
> [mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext 
> Aniello Del Sorbo
> Sent: 24 September, 2009 12:20
> To: Gil Quim (Nokia-D/Helsinki)
> Cc: maemo-developers@maemo.org
> Subject: Re: How to use extras-testing correctly?
> 
> I personally think we're taking a wrong approach to the problem.
> 
> I am not developing for  Apple because of their approval 
> process and their desire of control.

That is a different matter alltogether. One device, one shop, complete control. 
That has very little to do with a community site like maemo.org and a deevice 
using debian repositories.

> I don't want my application to go through an approval process 
> to reach my users.

No one is forcing you to place your app in extras. However it is the place most 
likely to be found by new users.
Maemo Extras should be of high quality because we will have completely new 
users to the platform. We don't want to scare them away, simple as that.

> I think that the developer HAS the last work on whether or 
> not her application is ready to be promoted to Extras.

The developer has the last word; just don't promote from testing.

> But I like the idea of my application being tested and I'd 
> love to get feedback from the final users.
> Android has a nice approach, I think the application goes 
> straight to Market and when a user removes it, she's being asked why.

I don't think so, please provide a reference.

> I learned that the extremes are almost never the best choice.
> Thus, I would like to see is a mix of Apple idea of "testers" 
> and Google Android "uninstallation survey".
> 
> Here's my proposal:
> 
> We leave Extras-Testing as it is, with votes, comments, bug 
> reports an so on (or improve it, as you wish).
> But then it's only the developer that decides, based on that 
> feedback, if or not to promote the application to Extras.

Which is how it works, except you need ten thumbs up.

> When the application gets promoted to Extras, people will 
> start using it.

Yes, extras-testing for testing, -devel for development, extras for prime time.

> If there's a major bug, the user will uninstall the application.

Certainly.

> The Application Manager should ask the user, as Android 
> Market does, WHY she uninstalling the application.
> Of course she may choose not to give any feedback.
> One of the choice she has is "Uninstalling because of a bug" 
> and she can fill in the bug if she wish so.
> 
> This bug goes straight to bugzilla for that application and 
> we should count them and decide on two limits: a soft 
> threshold and a hard one.
> 
> Soft threshold:
> if the application goes below this soft limit a yellow flag 
> is raised (how to handle it, it's another topic).
> 
> Hard threshold:
> if the application goes below this hard limit a red flag is 
> raised and application pulled from Extras.

Look, if you want to remove stuff, it's better to do it early than late. The 
assumption is that someone will submit bad code, then the better option is to 
take it out before it hits peoples devices. Not after it's gotten to the public.

> This way the Developer has full control of when and how she 
> wants her application to reach the public.
> The Community has some control of which applications should 
> be removed.

Which is there already. When you get the ten votes on testing, you get notified 
and can choose to ignore it or promote when you feel like it.

> A good application with a good developer stays there.
> A good application with a slow developer might get a yellow 
> flag and developer may feel pushed more to fix bugs A good 
> application with a bad developer might become buggy over 
> time, outdated and gets pulled out A bad application with a 
> good developer has a better chance to become a good one A bad 
> application with a slow developer has its life on the edge of 
> a razor.. be ware A bad application with a bad developer is 
> dead and not in Extras.
> 
> Aniello
> 
> 2009/9/24 Quim Gil :
> >
> >
> > ext Fred Lefévère-Laoide wrote:
> >> I know you can't force people to vote or test ... But it 
> means that 
> >> if an app has a very limited public it doesn't get out of 
> extras-testing ?
> >
> > This is a very good point. I believe it won't be an issue 
> once there 
> > are more people with devices able to vote, but still.
> >
> > Something that would help is to have a strict chronological 
> sorting at 
> > 
> http://maemo.org/packages/repository/qa/fremantle_extras-testing_free_
> > armel/ so oldest sub

Re: How to use extras-testing correctly?

[Henrik Hedberg]

Niels Breet wrote:

> On Thu, September 24, 2009 11:19, Aniello Del Sorbo wrote:
>> Here's my proposal:
>>
>>
>> We leave Extras-Testing as it is, with votes, comments, bug reports an
>> so on (or improve it, as you wish). But then it's only the developer that
>> decides, based on that feedback, if or not to promote the application to
>> Extras.
>>
> This is what happens now. Even if your app gets enough votes for promotion
> to Extras, you still need to push the button. There is no automatic
> promotion.
> 
> The developer/maintainer is always the one who decides on when to publish
> the app.

So no quarantine, no need to gather karma points? Please, update the 
wiki [1], if that will be the case.

Aniello was saying that developer should have an option to promote 
his application directly into extras regardless of the extras-testing 
QA. I second to that.

In addition, Aniello had nice ideas about the processes and tools 
how to find applications (and developers) who had probably misused their 
power to promote application directly into extras. There is no hurry to 
implement those, although they could make sense in the long run.

BR,

Henrik

[1] http://wiki.maemo.org/Extras-testing#Promotion_.2F_Demotion

-- 
Henrik Hedberg  -  http://www.henrikhedberg.net/
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Niels Breet]

On Thu, September 24, 2009 11:19, Aniello Del Sorbo wrote:
> Here's my proposal:
>
>
> We leave Extras-Testing as it is, with votes, comments, bug reports an
> so on (or improve it, as you wish). But then it's only the developer that
> decides, based on that feedback, if or not to promote the application to
> Extras.
>
This is what happens now. Even if your app gets enough votes for promotion
to Extras, you still need to push the button. There is no automatic
promotion.

The developer/maintainer is always the one who decides on when to publish
the app.

The community QA testing is there to find trivial issues, complex problems
might not be caught. But at least you can't promote an obviously broken
package by mistake.

Let's not make this too complex, before you know it we have more
bureaucracy than the often hated stores.


--
Niels Breet
maemo.org webmaster


___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Aniello Del Sorbo]

I personally think we're taking a wrong approach to the problem.

I am not developing for  Apple because of their approval process and
their desire of control.
I don't want my application to go through an approval process to reach my users.
I think that the developer HAS the last work on whether or not her
application is ready to be promoted to Extras.

But I like the idea of my application being tested and I'd love to get
feedback from the final users.
Android has a nice approach, I think the application goes straight to
Market and when a user removes it, she's being asked why.

I learned that the extremes are almost never the best choice.
Thus, I would like to see is a mix of Apple idea of "testers" and
Google Android "uninstallation survey".

Here's my proposal:

We leave Extras-Testing as it is, with votes, comments, bug reports an
so on (or improve it, as you wish).
But then it's only the developer that decides, based on that feedback,
if or not to promote the application to Extras.

When the application gets promoted to Extras, people will start using it.

If there's a major bug, the user will uninstall the application.

The Application Manager should ask the user, as Android Market does,
WHY she uninstalling the application.
Of course she may choose not to give any feedback.
One of the choice she has is "Uninstalling because of a bug" and she
can fill in the bug if she wish so.

This bug goes straight to bugzilla for that application and we should
count them and decide on two limits: a soft threshold and a hard one.

Soft threshold:
if the application goes below this soft limit a yellow flag is raised
(how to handle it, it's another topic).

Hard threshold:
if the application goes below this hard limit a red flag is raised and
application pulled from Extras.

This way the Developer has full control of when and how she wants her
application to reach the public.
The Community has some control of which applications should be removed.

A good application with a good developer stays there.
A good application with a slow developer might get a yellow flag and
developer may feel pushed more to fix bugs
A good application with a bad developer might become buggy over time,
outdated and gets pulled out
A bad application with a good developer has a better chance to become a good one
A bad application with a slow developer has its life on the edge of a
razor.. be ware
A bad application with a bad developer is dead and not in Extras.

Aniello

2009/9/24 Quim Gil :
>
>
> ext Fred Lefévère-Laoide wrote:
>> I know you can't force people to vote or test ... But it means that if
>> an app has a very limited public it doesn't get out of extras-testing ?
>
> This is a very good point. I believe it won't be an issue once there are
> more people with devices able to vote, but still.
>
> Something that would help is to have a strict chronological sorting at
> http://maemo.org/packages/repository/qa/fremantle_extras-testing_free_armel/
> so oldest submissions are always on top. And encourage people to help
> kicking the apps on the top even if they don't use it normally. At the
> end it's not that hard to give an app a shot and look whether the basic
> QA factors are in place.
>
> I believe after the N900 sales start there will be also a sustainability
> approach for the applications not used by many.
>
> Some will be highly specialized with a small but committed audience that
> will help testing and ranking. If a bug goes through they will be the
> ones suffering it first.  :)
>
> Some will be just starting and have a real small audience. But well, in
> such cases a bug going through won't cause much harm either. In most
> cases as the audience grows the less obvious bugs will pop up and then
> the developer will need to respond with newer and better versions. If
> major bugs are standing without fixes and the users of the app increase,
> then the relevance of the bug will increase as well and it might imply
> at some point the demotion back to extras-testing.
>
>
> --
> Quim Gil
> open source advocate
> Maemo Devices @ Nokia
>
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>



-- 

--
anidel
Sent from London, Eng, United Kingdom
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Quim Gil]

ext Fred Lefévère-Laoide wrote:
> I know you can't force people to vote or test ... But it means that if 
> an app has a very limited public it doesn't get out of extras-testing ?

This is a very good point. I believe it won't be an issue once there are
more people with devices able to vote, but still.

Something that would help is to have a strict chronological sorting at
http://maemo.org/packages/repository/qa/fremantle_extras-testing_free_armel/
so oldest submissions are always on top. And encourage people to help
kicking the apps on the top even if they don't use it normally. At the
end it's not that hard to give an app a shot and look whether the basic
QA factors are in place.

I believe after the N900 sales start there will be also a sustainability
approach for the applications not used by many.

Some will be highly specialized with a small but committed audience that
will help testing and ranking. If a bug goes through they will be the
ones suffering it first.  :)

Some will be just starting and have a real small audience. But well, in
such cases a bug going through won't cause much harm either. In most
cases as the audience grows the less obvious bugs will pop up and then
the developer will need to respond with newer and better versions. If
major bugs are standing without fixes and the users of the app increase,
then the relevance of the bug will increase as well and it might imply
at some point the demotion back to extras-testing.


-- 
Quim Gil
open source advocate
Maemo Devices @ Nokia

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Fred Lefévère-Laoide]

And you're lucky if you get votes !

I can imagine nobody cares about pwsafe (except 2 people) but does that 
mean that if you get no vote in extras-testing you'd better forget about 
your app and start something new ?

I know you can't force people to vote or test ... But it means that if 
an app has a very limited public it doesn't get out of extras-testing ?

Fred


Henrik Hedberg a écrit :
> Extras-testing QA is not working as it is implemented now! There are
> two main issues:
> 
> * Comments are stored into a wrong place. Those belong to Bugzilla! It 
> is double effort for a developer to track two different places or to
> transfer reports into bug tracking system manually.
> 
> * Developers are giving karma based on their subjective thinking instead
> of agreed (?) QA requirements.
> 
> Let's analyse karma and comments that Mauku has got:
> 
> * Two testers of five either add a new bug report or search the existing 
> bug reports before entering a comment. Good for those two, bad for the rest.
> 
> * Negative karma given at 2009-09-24 04:52 UTC and related comment 
> written at 2009-09-24 04:55 UTC. Is there any real reason to give 
> negative karma based on the comment? Tester either wants some new 
> features (definitely not based on QA requirements) or some minor user 
> interface modifications (not a show stopper).
> 
> * All testers report at least one issue that is not actually related to 
> the application itself but the underlying library. I understand that it 
> is hard for end-user to see the difference, but what happens when the 
> library is updated? These issues are fixed, and so is the application 
> also, but the negative karma stays there.
> 
> My suggestion here is that:
> 
> * Negative karma can be given _only_ if it based on the agreed QA 
> requirements.
> 
> * The package page should have a link to a bug tracker.
> 
> * Negative karma can be given _only_ with a link to a bug tracker having 
> a bug report about the show stopper. It may be either a new bug report 
> written by the tester or an old open bug report just referred in the 
> comment.
> 
> * Negative karma is automatically removed when the related bug report is 
> closed (fixed or other way resolved).
> 
> Graham Cobb wrote:
> 
>> Testers should be testing against the agreed
>> requirements only.  The subjective element should be eliminated as
>> much as possible - we are using human testers because it is
>> impossible to do this testing manually, not because we expect
>> different opinions.  We are using more than one tester just so that a
>> problem doesn't slip through because one tester missed it, not
>> because we expect voting.
> 
> I strongly agree!
> 
>> By all means add a comment if you are unhappy with the UI but it
>> should get a +1 as long as it doesn't conflict with the requirements.
>> Ideally every -1 should require a comment saying which of the QA
>> requirements is violated.
> 
> I strongly agree. Actually, I would remove the word "ideally". It is 
> a must! And with the relationship to bug tracker I described above.
> 
> BR,
> 
> Henrik
> 

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Henrik Hedberg]

Extras-testing QA is not working as it is implemented now! There are
two main issues:

* Comments are stored into a wrong place. Those belong to Bugzilla! It 
is double effort for a developer to track two different places or to
transfer reports into bug tracking system manually.

* Developers are giving karma based on their subjective thinking instead
of agreed (?) QA requirements.

Let's analyse karma and comments that Mauku has got:

* Two testers of five either add a new bug report or search the existing 
bug reports before entering a comment. Good for those two, bad for the rest.

* Negative karma given at 2009-09-24 04:52 UTC and related comment 
written at 2009-09-24 04:55 UTC. Is there any real reason to give 
negative karma based on the comment? Tester either wants some new 
features (definitely not based on QA requirements) or some minor user 
interface modifications (not a show stopper).

* All testers report at least one issue that is not actually related to 
the application itself but the underlying library. I understand that it 
is hard for end-user to see the difference, but what happens when the 
library is updated? These issues are fixed, and so is the application 
also, but the negative karma stays there.

My suggestion here is that:

* Negative karma can be given _only_ if it based on the agreed QA 
requirements.

* The package page should have a link to a bug tracker.

* Negative karma can be given _only_ with a link to a bug tracker having 
a bug report about the show stopper. It may be either a new bug report 
written by the tester or an old open bug report just referred in the 
comment.

* Negative karma is automatically removed when the related bug report is 
closed (fixed or other way resolved).

Graham Cobb wrote:

> Testers should be testing against the agreed
> requirements only.  The subjective element should be eliminated as
> much as possible - we are using human testers because it is
> impossible to do this testing manually, not because we expect
> different opinions.  We are using more than one tester just so that a
> problem doesn't slip through because one tester missed it, not
> because we expect voting.

I strongly agree!

> By all means add a comment if you are unhappy with the UI but it
> should get a +1 as long as it doesn't conflict with the requirements.
> Ideally every -1 should require a comment saying which of the QA
> requirements is violated.

I strongly agree. Actually, I would remove the word "ideally". It is 
a must! And with the relationship to bug tracker I described above.

BR,

Henrik

-- 
Henrik Hedberg  -  http://www.henrikhedberg.net/
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Quim Gil]

ext Graham Cobb wrote:
> By all means add a comment if you are unhappy with the UI but it
> should get a +1 as long as it doesn't conflict with the requirements.
> Ideally every -1 should require a comment saying which of the QA
> requirements is violated.

Agreed.

And even better, a link to a bug report with Major severity or worse.

-- 
Quim Gil
open source advocate
Maemo Devices @ Nokia
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Graham Cobb]

My long reply was lost when my N900 crashed while writing it.  So this will 
have to do.

I strongly disagree.  Testers should be testing against the agreed requirements 
only.  The subjective element should be eliminated as much as possible - we are 
using human testers because it is impossible to do this testing manually, not 
because we expect different opinions.  We are using more than one tester just 
so that a problem doesn't slip through because one tester missed it, not 
because we expect voting.

By all means add a comment if you are unhappy with the UI but it should get a 
+1 as long as it doesn't conflict with the requirements.  Ideally every -1 
should require a comment saying which of the QA requirements is violated.

Graham

--
Sent from Nokia Nseries mobile computer
- Original message -
> Am Mittwoch, den 23.09.2009, 11:07 +0100 schrieb Andrew Flegg:
> > It's easy to find problems, but our basic QA procedure started off as
> > "installs cleanly, uninstalls cleanly, doesn't waste battery, has
> > human readable description, has nice icons" and now "uses /opt and
> > doesn't waste disk space if appropriate". These are detailed at:
> >
> >        http://wiki.maemo.org/Extras-testing#Quality_criteria
> >
> > Should someone be pressing thumbs up if a package meets those
> > requirements and has no blocking/crashing bugs? That doesn't seem to
> > be happening - is it because no-one's testing, or because no-one feels
> > clear as to what they should be doing?
>
> I must admit that so far I've considered giving a Thumbs Up or Thumbs
> Down to be a purely subjective decision based on personal impression.
> I for myself e.g. would give a Thumbs Down for a very small, working app
> if the main menu is not fremantle'ized (as the maintainer does not have
> a large codebase to maintain this should not be too hard), but that
> would not be much of an issue to me for a large complicated app (lots of
> other more important stuff to maintain and work correctly).
>
> That's why others can still vote +1 if I said -1 and the other way round
> - I assume everybody has his/her own ideas about thresholds and
> expectations and that's exactly why there's more than one person allowed
> && required to vote.
>
> Just my current point of view,
> andre
> --
> Andre Klapper (maemo.org bugmaster)
>
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Andre Klapper]

Am Mittwoch, den 23.09.2009, 11:07 +0100 schrieb Andrew Flegg:
> It's easy to find problems, but our basic QA procedure started off as
> "installs cleanly, uninstalls cleanly, doesn't waste battery, has
> human readable description, has nice icons" and now "uses /opt and
> doesn't waste disk space if appropriate". These are detailed at:
> 
> http://wiki.maemo.org/Extras-testing#Quality_criteria
> 
> Should someone be pressing thumbs up if a package meets those
> requirements and has no blocking/crashing bugs? That doesn't seem to
> be happening - is it because no-one's testing, or because no-one feels
> clear as to what they should be doing?

I must admit that so far I've considered giving a Thumbs Up or Thumbs
Down to be a purely subjective decision based on personal impression.
I for myself e.g. would give a Thumbs Down for a very small, working app
if the main menu is not fremantle'ized (as the maintainer does not have
a large codebase to maintain this should not be too hard), but that
would not be much of an issue to me for a large complicated app (lots of
other more important stuff to maintain and work correctly).

That's why others can still vote +1 if I said -1 and the other way round
- I assume everybody has his/her own ideas about thresholds and
expectations and that's exactly why there's more than one person allowed
&& required to vote.

Just my current point of view,
andre
-- 
Andre Klapper (maemo.org bugmaster)

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Andrew Flegg]

On Wed, Sep 16, 2009 at 14:22, Graham Cobb  wrote:
>
> The bottom line is that we need to try to use the process and see
> how well it works.

I've got a concern on the other side. There still seems to be
murkiness about what our processes are here - and these will evolve
through time, as you say.

For example, Tero's added a comment on the latest version of Attitude:

   
http://maemo.org/packages/package_instance/view/fremantle_extras-testing_free_armel/attitude/0.0.8

Is that enough to stop someone clicking the thumbs up? Should it be?
Will anyone click it if no-one else has?

It's easy to find problems, but our basic QA procedure started off as
"installs cleanly, uninstalls cleanly, doesn't waste battery, has
human readable description, has nice icons" and now "uses /opt and
doesn't waste disk space if appropriate". These are detailed at:

http://wiki.maemo.org/Extras-testing#Quality_criteria

Should someone be pressing thumbs up if a package meets those
requirements and has no blocking/crashing bugs? That doesn't seem to
be happening - is it because no-one's testing, or because no-one feels
clear as to what they should be doing?

Is it just a case of the "Quality criteria" being more clearly
communicated? Perhaps the bullet list of points should be embedded in
the /packages/ UI?

Cheers,

Andrew

-- 
Andrew Flegg -- mailto:and...@bleb.org  |  http://www.bleb.org/
Maemo Community Council chair
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[tero.kojo]

- Original message -
>
> tero.k...@nokia.com wrote:
>
> > Once there is hardware available, I think that the limit is the
> > quaranteen time that the app has to stay in extras-testing. Knowing the
> > community, my guess is that there will be enough testers for the apps.
> > (some people code well, others like to test)
>
> Is it possible to predict or set the actual moment when a package
> goes into extras?  For example, if I liked to make a grand announcement
> of my new software, how could I time my announcement when the control of
> the release date and time is not in my own hands?

I understood the process to be such that the package owner get's notified of 
the pass, but can/has to press the promotion button to extras. That would 
enable the grand entrance with fanfare and timing.

Niels can correct me if I'm lost on this one.

Tero

> BR,
>
> Henrik
>
> --
> Henrik Hedberg - +358 (0)40 574 5087 - http://www.henrikhedberg.net/
> Innologies - Innovative Technologies - http://www.innologies.fi/
> Oulu, Finland - FI19934487, VAT reg. - http://www.innologies.com/
> ___
> maemo-developers mailing list
> maemo-developers@maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
>

___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Henrik Hedberg]

tero.k...@nokia.com wrote:

> Once there is hardware available, I think that the limit is the 
> quaranteen time that the app has to stay in extras-testing. Knowing the 
> community, my guess is that there will be enough testers for the apps. 
> (some people code well, others like to test)

Is it possible to predict or set the actual moment when a package 
goes into extras?  For example, if I liked to make a grand announcement 
of my new software, how could I time my announcement when the control of 
the release date and time is not in my own hands?

BR,

Henrik

-- 
Henrik Hedberg - +358 (0)40 574 5087 - http://www.henrikhedberg.net/
Innologies - Innovative Technologies - http://www.innologies.fi/
Oulu, Finland - FI19934487, VAT reg. - http://www.innologies.com/
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

Re: How to use extras-testing correctly?

[Graham Cobb]

On Wednesday 16 September 2009 13:15:59 Till Harbaum wrote:
> 1) don't update too often, so people/testers don't get bored
> 2) put some extensive internal testing before promoting something
>   to extras-testing

My conclusions are similar but maybe a little different.

I am not promoting GPE to extras-testing until I have done some testing in 
extras-devel.  But I am not waiting for the app to be of releasable quality 
before I promote it.   I have divided the problems I know about into 3 
categories:

1) things which have to be fixed before going into extras-testing as they 
either make the app non-functional or carry some risk for the tester (e.g. a 
bug preventing switching to another app)

2) things which are high priority to fix and will probably be commented on by 
testers but which do not prevent doing useful testing

3) things which may not be fixed for the first release

I will promote once all the problems in category 1 are fixed.

> This also means that i will delay the promotion of  osm2go as i am
> currently running my own tests.

So, I am delaying promotion until I think the application is (i) usable, and 
(ii) not putting testers at risk of major problems.  This involves some 
amount of both testing and bugfixing -- probably a delay of about a couple of 
weeks in my case.

> I somehow think this is not the way it should work ... Any ideas how to use
> it in a more efficient way?

I do share your concern about how the frequency of updates will work.  Once I 
promote, and people do some testing and find problems which I need to fix, 
how often do I update?  If I do it one-by-one for each report, testers may 
find themselves testing the same features several times in one week!  My plan 
is to try to update about once a week -- but that may mean quite slow 
progress.  

The bottom line is that we need to try to use the process and see how well it 
works.

Graham
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers

RE: How to use extras-testing correctly?

[tero.kojo]

From: maemo-developers-boun...@maemo.org 
[mailto:maemo-developers-boun...@maemo.org] On Behalf Of ext Till Harbaum
Sent: 16 September, 2009 15:16
To: maemo-dev
Subject: How to use extras-testing correctly?

Hi,

i have been asked to upload osm2go to extras-testing. It's not quite clear to
me how this is actually supposed to work. I see several possibilities:

1) Just upload something. Others will do some testing and i can upload
 a fixed version of they see problems. The new version will then go though
 the same testing again. This is how i think it was initially meant to work.

2) Upload only things that have gone through my own testing to increase
  the likeliness that it will go straight through testing without any problems.
  This is the way i will use it.

I see several problems with 1):

- If something broken doesn't pass the tests and i keep uploading new
  versions people will likely get tired of me and my perhaps perfect
  version will stay in testing forever because nobody wants to test my
  app over and over again

I rather see it as a chance to try out the latest versions. And as a tester a 
way to do my bit for the community. I'm not an excellent developer, but I can 
test applications.

And you are right, broken stuff shouldn't pass into the hands of the average 
high end consumer (because yes, the N900 will not only be in the hands of oss 
hackers, but a lot of other people). That's why there's the testing phase.

People most likely get tired if the application is of no use and broken. A 
repeated upload of a broken "Hello world", might get disregarded soon. But 
something that people consider usefull and that has been downloaded over 10 000 
times... A few broken versions in the middle will hardly matter.
Actually it would be worse to push those broken versions directly to extras, 
that would hurt all those people, not just the testers.

The social dynamics of the system will make sure that things get tested. If an 
app gets a vote (up or down), others are likely to out of curiosity see what 
the tester said, and maybe even try it out themselves.

- If something broken actually passes the testing things get even worse
  as a bug fix has to go through testing again and the broken version will
  stay there until the bug fix passes testing

Some bugs will go through naturally. Perfect software doesn't exits. Don't even 
think so.
The thing is that testing weeds out bugs that would be considered blockers or 
major issues. I can live with a lot of minor issues, and I won't even notice 
them most of the time. When I do, I'll report them as bugs.

To me this means two things:

1) don't update too often, so people/testers don't get bored

Keeping changes to yourself increases the likelyhood that there are issues that 
you don't find. The author of the software is never the best tester. The best 
tester is completely independent, something which strangely enough we can 
achieve in the community.

Release often and release early. It does not say release broken.
Early so that the tester knows what is coming in. The worst testing experience 
will be the first time; new software, massive feature list and no base on which 
to work.
Often so that the tester can check the changed part fast and that the delta 
doesn't grow too much from the previous version.

2) put some extensive internal testing before promoting something
  to extras-testing

Sure, but which one do you like more, developing or testing?
If you are one of the few people (I know two) who utterly love both, then test 
until you think the test is exhaustive. Then give out and let others test too.

However my guess (and experience) is that people who write oss software love 
developing more than testing. Extras-testing provides testing as a service to 
the developer.

This also means that i will delay the promotion of  osm2go as i am currently
running my own tests.

Well, right now is a special time. There is practically no hardware available 
(some developer units in the hands of Nokia personnel and a few selected 
people). Right now it is hard to get ten votes at all. That just happened 30 
minutes ago for the first time (rootsh 1.5 is considered good for extras, a 
fitting package to go first).

And there is no stopping people from keeping their stuff in extras-devel for as 
long as they want.

Once there is hardware available, I think that the limit is the quaranteen time 
that the app has to stay in extras-testing. Knowing the community, my guess is 
that there will be enough testers for the apps. (some people code well, others 
like to test)

I somehow think this is not the way it should work ... Any ideas how to use
it in a more efficient way?

The idea is to make sure that the end user gets good software.

Tero

Till


___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers