Am 08/14/2013 09:58 PM, schrieb Rob Weir:
On Wed, Aug 14, 2013 at 3:37 PM, Marcus (OOo)<marcus.m...@wtnet.de> wrote:
Am 08/14/2013 09:01 PM, schrieb Raphael Bircher:
Am 14.08.13 20:21, schrieb Rob Weir:
On Wed, Aug 14, 2013 at 1:36 PM, Edwin Sharp<el...@mail-page.com> wrote:
Dear Rob
The 4.0 release was too ambitious - we should advance in smaller steps.
Nothing compares to general public testing - betas and release
candidates should not be avoided.
TestLink cases should be less comprehesive (in terms of feature
coverage) and more stress testing oriented.
The number to consider here is how many defects were found and fixed
during the 4.0.0 testing, before the general public users had access?
I assume it was quite substantial. If so, the TestLink usage was
effective. In other words, we might have found fewer bugs without
using it.
In general, my feeling is that it's too early to do a retrospective and ask
"what was good, what needs to be improved?" (to say it with SCRUM words ;-)
) Just after the first major release.
Memories are still fresh and programmers are looking at the relevant
code. This is the best time to answer these questions.
We should look on the BZ query you have mentioned and see if there is one or
more hotspots that should be improved fast. That's it.
That would be fine. I'm not suggesting anything radical. Gradual,
but constant improvements are the way to high quality.
Then this should be sufficiant IMHO.
This is important to keep in mind: we want to prevent or find more
bugs, but we're not starting from zero. We're starting from a process
that does a lot of things right.
I like the idea of a public beta. But consider the numbers. The 40
or so regressions that were reported came from an install base (based
on download figures since 4.0.0 was released) of around 3 million
users. Realistically, can we expect anywhere near that number in a
public beta? Or is it more likely that a beta program has 10,000
users or fewer? I don't know the answer here. But certainly a
well-publicized and used beta will find more than a beta used by just
a few hundred users.
The public beta is from my point of view realy important. Even you have
only 10'000 Downloads of a beta, you have normaly verry experianced
Users there, like power users from Companies. They provide realy valua
feedback. So from my point of view, this is one of the moast important
changes we have to do. For all Feature release a beta version. And don't
forget, people are realy happy to do beta tests. but many of them are
maybe not willing to follow a mailing list.
A public beta release is of course not the golden solution but could
activate some power users that give us the feedback we want and need.
So, +1 for going this way.
After the 4.1 release is done we can see if this was much better - and ask
ourself why? :-)
But several of the bugs in 4.0.0 should never have made it to even a
beta. For example, the very slow saving in Excel format. What
happened there?
Sure, this could be fixed later in the process, in a beta, or in a
4.0.1 as we're doing now. But this really should have been detected
and fixed much earlier in the process.
What are betas good for? Betas are good for expanding the set of
configurations. Platforms, languages, extensions, etc. But the
informal "tests" done by real users tend to be shallow. Also, we have
no way of determining what the test coverage is. In particular we
have no basis for telling the difference between the coverage of a 2
week beta versus a 4 week beta. But with out formal test cases we can
easily look at how many test cases were executed. We could even look
at code coverage if we wanted to.
Maybe if there was a way to record what features beta testers were
using... Or even have a little survey where they report what
platform they ran on, etc.
But very slow saving to XLS files? A defect like that should have
been caught by us before a beta and before a RC. We shouldn't expect
to find bugs like that in a beta.
Of course. I've said nothing different. ;-)
Finally, I think we can all point to a similar open source project
that has numerous betas, but still suffers from poor quality. So a
public beta, by itself, is not sufficient. We need some upstream
improvements as well, I think. But we should do a beta as well. But
aim to have the highest quality beta we can, right?
Sure, the XLS bug has nothing to with "this wouldn't happen with a
Beta". As I answered to Hagar this is one hotspot that should be looked
at closer.
Marcus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org
