Michael Scherer wrote:
Well, from a physical point of view, everything is limited, so saying
"limited ressources" didn't indeed told much.
I think that the ressources at Mandriva could be summarized as "around 1
to 3 full time people ( maybe more, maybe less, and likely not full time
on the stable free distro )".
Which is not balanced at all when compared to the ressources that were
placed in packaging. Ie, there was much more people to produce rpm than
people to
1) take care of update ( secteam, 2 people )
2) take care of testing update ( qateam, 1-3 people )
Being unbalanced lead to the main/contribs split with the complexity and
problem that went with it. Of course, the goal is not to have less
packagers, but rather more Qa people, the 2 being not exclusive.
This then bring to the simple question is "why did we have more
packagers than QA ?"
My own opinion is that because packaging was opened to external
contribution since almost the start ( since 10 years, packagers number
have growth ), while QA was not, and I suppose that was due to a lack of
time devoted on making QA more open ( ironically likely due to a lack of
ressources at the first place ).
And so, I think we are now in a totally different situation. QA will be
more open, because it cannot be closed. We can ( and I think we should )
make the QA ressources grow with the packagers one ( among others ).
So how can we do ?
While this may not seems apparent at first sight, I think that Fedora is
actually leading in term of community QA process ( we still had the lead
in term of automated QA ).
Basically, packages that are updated requires to be noted, with a system
of karma ( http://fedoraproject.org/wiki/Bodhi_Guide#Karma ).
Positive karma, the update is pushed, negative, it is not.
Anybody can test anything, even if there is also a proven tester group.
There is even the concept of critical path packages, aka very important
package that must be deeply tested
( http://fedoraproject.org/wiki/Critical_Path_Packages ).
Of course, the system is not perfect and will not solve everything. For
example, last week, openldap update broke server functionality :
( http://lists.fedoraproject.org/pipermail/devel/2010-November/146097.html )
But still, having community enabled QA is a great way to have every grow
properly.
And in fact, that's exactly one of the feature of your project
mageia-app-db. So we could indeed have a better QAteam by easing the
work of community, using mageia-app-db. Ie, take regular user, and turn
them in QA team member.
And so, doing like this would enable to give us :
- real involvement from some users
- balanced community, not overwhelmed by technical maniac packagers
( like me )
- having a better Qa, for a better system
So while nothing is done, while I am not a member of the QA team nor a
leader, while I just speak to voice my opinion, and while this is just a
proposal based on what other have done to solve the same issue than us,
this show that we can have more ressources than what Mandriva had.
why this kind of "competition"?
Ie, we need to think with a fresh mind, and I am sure that people can
creatively propose solutions on the problem :
"how can we have a more balanced QA and packagers team".
What you are saying is good, and I would like to extend what you posted
with what follows. IMHO we are trying to focus too much on what it was
wrong to forget what it was already good, like if what was already good
in the distro would be granted by default (e.g. trying to focus too much
on what is already in ANY of the 100 distrowatch.com distros: all of
them have the same core-packages on the other hand] to forget what is
was already good or better: e.g. good support for extra hardware, better
support for scientific applications, etc.).
We should summarize what there is from what there isn't in the distro.
The 2nd problems is that we seem assigning to most packagers and
maintainers some secret powers that they should have, like knowing the
package better or at the same level than upstream developers. If not,
then there are lack of resources...; maybe they just know how to
assemble and compile it...and that's all. IMHO knowing the package at
such high levels has become the exception, not the rule. One
packager/maintainer might know very very well 1 or maybe 3,4 packages
(e.g. colin for pulseaudio, buchan for samba, me for tex and so on), but
there are 10000 packages out there and there aren't 5000 maintainers...
I give you an example. In MDV 2010.0 or 2010.1 the kdenlive package, a
video editing application, was not working at all. It crashes on basic
stuff as soon as you open or import a video file. In other words
unusable. How was possible that? Well, one might say poor QA, bad
packagers, bad maintainers or no maintainers at all, or no QA at all. It
your fault, it's their fault, it's our fault. Indeed it doesn't matter.
The main problem is that we rely as main source of QA the bugzilla. So
what is reported on bugzilla has a bug that we should fix, what there
isn't should work by default and has automagically an higher QA score.
IMHO this is a false assumption. I might find it crashes and be lazy and
not doing any bugzilla report, or I'm not sure about what to tell
exactly, because maybe I think it's fault of my configuration, and so
on. In the specific example backports have a working kdenlive (maybe
it's just luck, dunno) but according to the supposed QA score the
kdenlive in main should be better than the one in backports. And indeed
it isn't because any program barely working is better than any program
crashing on startup.
To solve this there are a lot of solution, but it's not said that the
one or the other should influence people to use this distro and not
another, like we might think. E.g.
1) We might move kdenlive from good main to evil contrib or remove
kdenlive from the distro? Well, a non-existing package won't crash. And
so distro would result more polished, ordered (I like ordered and
cleaned distro with no broken deps in hdlists, really) should rock. But
from point of view of an end user he don't have any video editing
application anyway. So who cares of the others packages he won't use?
2) We might let people know that the backport version works. So more
communcation here.
3) We might add several new rules and weights on the shoulders of the
maintainers/packagers. A packager could fly away, so letting these rules
on the shoulders of the other survived packagers which then will be more
overloaded, and then could fly away more. Sound a bit like the
"highlander" packager...
4) We might help to let these rules easier, lighter and quicker to be
implemented (more communcation, self or packagers helping each others,
more automatic tools)
5) Add a precise protocol of testing for each package. This might be
automatic or manual (or both). In case of manual, the testing protocol
steps might be performed by anyone. But shouldn't go in the destructive
spiral of more rules. Documentation should be crystal clear, and
information should be easy to catch not let users waste days and days
trying to find the right thing to do, in seeking good documentation
between tons of obsolete and bad one. Also the report of the testing
should be easy to post, not opening 27 sites, scroll between 18000
packages, wait 10 minutes the server answers, or sending 84 mails and
wait answer from 54 different people...; In case of kdenlive, a basic
protocol test could be like this: 1) install the package 2) Go on menu
Project/Add clip and open the file blabla.avi HERE... 3) Go on menu
Monitor/Clip and click on the button "play" 4) ... 5) rely also on the
upstream protocol testing if exists 6) report it works by a couple of
clicks. This won't require specific skill and even users who might not
have ever used this package could partecipate.
5) Punish the one who uploaded the package. Blocking it for one lap.
Well this is a bit scaring ;-) Don't do that.
6) Any combination of the above.
Second problem is that there also any referring to documentation
support, but speak only from point of view of packagers. We might have
things that are rock solid, but if nobody knows how to use them... ;-)
Bye
Giuseppe.
