Michael Meeks wrote:
> Hi Mathias,
>
> Once again thank you for your thought provoking mail.
>
> On Mon, 2006-10-30 at 23:57 +0100, Mathias Bauer wrote:
>> You mix up some things here. Nobody said that we need a spec for each
>> and every "tiny ergonomic fix". We need them for new features - e.g. a
>> quickstarter on Linux. :-P
>
> I refer you to the Sun rubric ** emphasis added.
>
> http://wiki.services.openoffice.org/wiki/Category:Specification
>
> "I Want to Change Something in OpenOffice.org - Do I Have to
> Write a Software Specification?
> **In general the answer is YES**. This applies to:
> Features, Enhancements, **Defects**"
You should quote the whole text: ;-)
# Defects requiring the following type of changes:
* Behavioral changes of the UI (e.g. changing a dialog from modal to
modeless)
* Visual changes of the UI (e.g. changing the icon size, the splash
screen, the about box)
* Configuration changes (e.g. changing application defaults such as
Spellchecking ON/OFF)
This looks different. There is no general rule that a bugfix for a
defect needs a spec. My personal opinion is that this list is not very
specific and leaves a lot of room for interpretation. This is per se a
neutral fact - neither positive nor negative. The negative touch comes
from the implemented practice - that's what needs to be improved if you
have negative feelings about it.
Besides that you should consider that nowadays a spec can be just a very
short snippet of text that describes the changes in understandable and
hopefully unambiguous terms. That's all. I know that earlier specs where
a "little bit" oversized but IMHO that has changed now.
And if people are afraid of creating a document from a template -
Christian already wrote that we can place specs in a wiki. If that might
help I'm all for having this pretty fast. I hope you agree that it is a
good thing that the necessary things are written down at least somewhere
where they can be found easily.
> Now, of course we could explicitly exclude more things from this, which
> would be good - but AFAICS - at least as of now, each tiny ergonomic
> change requires -at-least- a round-trip to the team lead.
Where is the problem with a contact to the project lead? IMHO this could
be a very important step to a better support that you demanded. If a
project lead knew about a particular patch or CWS that is in the works
he could help much easier and better in case something is stuck. He
could also help to form an iTeam early and make sure that the necessary
work of all participants gets aligned and planned. Work can go like
clockwork if you do it this way but can take much more time if you work
on your own and later ask people for help when you are finished. Working
in parallel shortens the time you need to get something integrated.
> I hack on a module I like to try and find these tests, I poke in
> 'workben' and I see very frequently stale/un-buildable/un-runable code,
> then I poke in qa/ and eg. in configmgr/qa/unoapi I see a makefile.mk I
> 'dmake' that, something happens and it barfs:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/openoffice/Runner
> dmake: Error code 1, while making 'ALLTAR'
>
> it appears broken out of the box.
>
> I would -Love- to have a good, standardised unit testing framework in
> place to add tests to, and let us re-factor code more aggressively with
> confidence. However - I just don't see anything here.
I think you already found some of them. I don't know why the OOo Runner
doesn't work in your environment. Would you mind to file an issue and
assign it to "cn"?
>> We are currently investigating how we can get faster tests, one
>> direction we are looking into is avoiding or at least reducing the
>> idle/sleeping times.
>
> Yep - of course, understanding why these idle times are there would be
> good I guess.
Of course that's the first step. :-)
Our plan is to start with this pretty soon (in 1 or 2 weeks when the
developer of the testtool is back from vacation).
>> Other ideas are welcome. My very personal opinion
>> is that we should have more API (code) based tests and less GUI testtool
>> based ones but I know that there are other opinions. Must be discussed.
>
> Well; of course from a 'community' perspective - I'm well up for adding
> in-source, in-CWS, standardized, fast unit-tests. eg. reading the calc
> 'R1C1' work recently, I was itching to write a test-suite, that would
> simply exercise this piece of the calc code & parse 100 formulae of
> varying complexity and validate that the results were correct.
>
> Unfortunately it's -really- difficult to do that.
That's the kind of input we should collect. Not in this thread of
course, but somewhere else. I've put it on my list and will come back later.
>> There's nothing wrong with doing unit and API tests in Java.
>
> As long as they are easy to run with some standard command, I don't
> much care what they're written in.
>
>> And UNO components don't make anything more complicated
>
> Au contraire - if you have built all of OO.o up to 'sc' (eg.) and you
> now want to write a very small, very fast unit test to exercise just the
> formula parsing piece - you have a nightmare. Somehow, you have to get
> OO.o alive enough to actually start, bootstrap etc. you need to build a
> custom .rdb file [ we have ugly unsustainable hacks in 'workben'
> directories around the place to do some of this ]. You can't even read a
> file in without having an huge types.rdb, services.rdb, a ton of paths
> set right and a big piece of boilerplate code etc. etc. :-) AFAICS it's
> a huge pain.
That's not the fault of UNO, the problem is created by the fact that the
code you are wanting to test doesn't run without the whole office. UNO
components can be instantiated and tested with a very small and simple
environment. Unfortunately we don't have that much components that could
be tested in this way. You can't test any code in e.g. Calc that depends
on sfx2, svx, vcl etc. without these libraries being present and in many
cases also not without these modules being initialized in some way (what
usually is done by other Ooo code in the startup).
SAL e.g. is a C++ library that has a complete set of C++ tests for
(nearly?) every exported function. It can be tested by a simple test
application - because the implementation is encapsulated enough.
As announced at OOoCon I have started to document some facts about the
Ooo code "jungle". Perhaps some knowledge about them would make it more
understandable why code testing is so hard in OOo. Unfortunately I have
to do a lot of other things currently, e.g. taking part in long
discussions about specs, testing etc. :-)
> Of course - if there was an existing small/light/simple infrastructure
> that tests could be easily added to, then I for one would write more
> unit tests: this gap has frustrated me on the 2/3 times I've actually
> wanted to sit down & write a block of test code. [ And really, arguably,
> people should be writing the torture tests as they write the code ].
I think we should discuss this somewhere else. If you are interested in
writing tests but didn't find a suitable infrastructure we should do
something against this.
>
> The other problem with UNO is you can only test what is exposed via.
> UNO, and that is not enough to torture the internals often.
Yes, UNO based tests can't be used for every code in Ooo. But I don't
see this as a problem. I never said that you should use UNO for all
tests. But again: the missing accessibility to testing procedures is not
caused by the usage of UNO, it's the internal implementation that sets
the limits.
>> Or is there anything you find more complicated in unit testing of UNO
>> components? Then please give an example.
>
> It's possible my horrific past experience of UNO bootstrapping is now
> obsolete :-) if so, wonderful. I'm looking for a solution that doesn't
> require "installation", can be run in the source tree very simply with
> 'dmake check' (eg.) and will run through a list of test modules, build
> them, execute them, and report their output; and wrt. VCL - having a
> live X/GUI connection, at least for now is just fine. Preferably being
> able to automate this fully (on each CWS before it's nominated) would be
> ideal.
Again the code itself stands in the way, not UNO. This test procedure
would be applicable for only a few C++ components as most of them depend
on other libraries, many of them even from being part of an installated
office application and some of them even don't work without some code
executed in the startup.
>> You again mix things here. This is no longer true for fixes in 2.0. And
>> nobody asks for specs for bug fixes. Please give examples where a bug
>> fix was not integrated because a spec was missing.
>
> It depends what you see as a bug. I see something being unusable as a
> serious bug, because I care about usability; OTOH you might call fixing
> that 'a feature' :-) Ultimately, the end-user sees it as a bug, and the
> customer must be always right - surely ?-)
Yes, good point. I will need to think about this. The discussion about
the Calc issue in this thread (started by the mail of Jim Watson) showed
that "both sides" have good arguments, but it seems that we are stuck as
long as nobody tries to see it in a different light. This reminds me on
a funny quote I found somewhere (forgot from whom it was):
"Before you criticize someone, walk a mile in their shoes.
That way when you criticize them, you'll be a mile away.
Also, you'll have their shoes."
>> The problem is that you don't know this beforehand - and in the past one
>> of our biggest problems was that we introduced code changes and later on
>> the master wasn't usable for weeks.
>
> Ok - I hate being clobbered with this -simply-appalling- practise of
> people randomly committing API breakage, and then expecting others to go
> around cleaning it up. AFAICS the vast majority of the breakage you
> refer to was related to this, and yes - this was (to my mind) a totally
> stupid way to work :-) [ hopefully we don't need to argue that
> through ].
It's not API breakages alone. But we don't need to stress this any
longer as you seem to agree that we should at least not integrate
anything as long as we have the valid suspect that it could break the
master in a way that many people can't use it, right? So we can
concentrate on other cases where you think that not integrating
something fast seems to be exaggerated if the chance that it will break
something looks neglectable.
> On the other hand, it's possible to keep the code building and running
> at all times, and yet commit lots of code changes; other projects manage
> to do this quite successfully. And obviously it's impossible to improve
> the quality without changing the code, in some cases really quite a
> lot :-)
I don't like these "other projects" arguments. This ends up in an apple
and oranges comparison. If you compare OOo with other projects you
should use projects with similar size, history and problems. You can't
compare OOo with a small project that e.g. was designed for being
testable from the beginning, projects without a GUI and many more.
It would be interesting to hear how projects like KOffice handle things
and what consequences it has. Of course we can also compare us with
another famous competitor though they do not work with open source. :-)
>> > OO.o should be of a lower quality in order
>> > to get great testing & feedback to improve the StarOffice quality; at
>> > least that is how I would structure it. Indeed - I am surprised that
>> > OO.o and StarOffice releases are ~concurrent (or that StarOffice
>> > sometimes leads) - that seems to me to be a recipe for poorer quality in
>> > StarOffice.
>>
>> I don't understand how you come to that crazy conclusion. Step back and
>> think about Thorsten words with an open mind and not with the explicit
>> will to use them against him.
>
> Nah - I don't have that will - but, I do hear (unsubstantiated) rumours
> of customer critical fixes getting rushed into the tree without the same
> level of process rigour that we all 'enjoy' ;-) [ perhaps that is now a
> defunct practise, I hope so ].
It was never a used practice anyway. You should check your sources. :-)
> Wrt. StarOffice / OpenOffice quality - there is certainly a possible
> strategy that makes sense here: to have somewhat different quality goals
> for StarOffice vs. OpenOffice. This is the approach with the Fedora/RHEL
> and OpenSUSE/SLE products - whereby the community product has a
> different balance: getting cool/trendy new features (with some
> instability) while the 'Enterprise' product has more stability and
> (perhaps) fewer features, or perhaps just arrives slightly later.
That's not our way of thinking. The community of OOo is completely
different to the Linux communities. There are much more "average Joe"
type users that have the same quality expectations as our StarOffice
customers. They are not looking for "cool/trendy" features, they want to
have something they can work with. And the same is true for Enterprise
customers. If they all would be fine with using (buying!) StarOffice
instead, well, I would sign that contract immediately. :-) But
unfortunately that's not the case. So do you opt for two different
versions of OOo (as you do with SuSE)?
>> And please accept that "throw everything in and fix the bugs later"
>> can't be the way to go. Been there, done that, felt the pain. Don't
>> want to be there again.
>
> Is that what I'm suggesting ? ultimately not; however - I suspect that
> this is a case of the pendulum swinging way too far the other way.
I don't deny that this is possible - though I'm not convinced that this
has happened. At least you have drawn my attention to this. :-)
> AFAICS what Sun experienced before was simply brain-damaged :-) I'm not
> suggesting committing stuff to HEAD that is -known- and -expected- to
> break other people, that is clearly folly. However, accelerating the
> pace of committing -fixes- and improvements is surely beneficial.
But as long as there is only one code line to work on you still have
problems with code that you think is fine but in fact contains a high
risk to break your master. This is a delicate balance and IMHO you can't
do a simple calculation that fixing as many bugs as possible
automatically will make the product better. It's more complicated.
>> Of course every rule can be changed if good arguments and (IMHO even
>> more important!) good alternatives are presented.
>
> Sure, so the suggestion of using the wiki for specs is positive - it
> removes a biggish barrier to entry, and (perhaps) makes it more useful.
> Some other constructive suggestions might be these:
>
> * duplicate as little state as possible:
> + no i18n data, no screenshots in specs
> + [ unless vital to illuminate some point ]
>
> * understand -precisely- what is required in a spec. and
> require no more than the minimum.
> + what is required is from my POV not well understood
>
> * focus the saving in (very scarce) developer / QA resource on
> other things: a good unit test framework eg.
>
> * re-direct some spec. process time in favour of peer code
> review - it will yield a higher quality, and ultimately
> better trained, more careful coders.
>
> * clearly specify the cases where full specifications are not
> required, and aim to extend this list
>
> * ensure that the spec. process is filled with shortish
> timeouts to handle people that don't respond
>
> * make every possible step asynchronous so there are as few
> round-trips as humanly possible
>
> * encourage stake-holders to be more responsive (IRC is a
> good tool here) so they can be consulted.
>
> * separate the 'team' aspect from the spec. working with an
> inter-disciplinary team is to generate -Design Requirements-
> and perhaps some UI design data is a useful thing, but
> there is no need to over-formalise that process.
Thanks for all this, this is valuable input. Finally we have put
together a lot of interesting things that enable me (and I hope some
others too) to make development @OOo a better experience. I think that
it doesn't make sense to prolong this thread by discussing it here. But
you can be sure that at least I will pick this up somewhere else and
make sure that we will have a discussion about this.
> Great. What I'd really like to hear is a clear articulation of the
> goals of the spec. process; what were the Design Requirements here ?
> also, what was the process of introducing the process ? apparently it
> just appeared suddenly, mandated from above, is that the case ?
I thought that this was addressed at the OOoCon already and perhaps also
in the specification project wiki. But in case not I hope that some of
the people that worked on the spec template (they already appeared in
this thread) can answer this.
> Ultimately though - wrt. the exasperated tone here - I can believe it's
> a problem, but can this be a surprise ? we have talked the issues over
> of streamlining this process in detail at the last 2 ESC meetings,
> reaching consensus each time on the same approach - once in March this
> year and again at OOoCon. Unfortunately as far as is humanly discernable
> from outside of Sun - progress in this area has totally stalled shortly
> afterwards. Now, it's great that you're responding & helping to unwind
> some of these problems.
Well, I think part of the problem is the harsh tone that some ;-) people
are using at times when it comes to specs. It prevents fruitful
discussions because it distracts them into flame wars.
Ciao,
Mathias
--
Mathias Bauer - OpenOffice.org Application Framework Project Lead
Please reply to the list only, [EMAIL PROTECTED] is a spam sink.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
